Why Your CRM Import Created Duplicates (and How to Stop It Happening Again)

Cause one: the import had no match key

A match key is the field the CRM uses to ask, “Is this person here yet?” Most CRMs let you pick it. Many imports skip that pick, or the person who runs the import does not know the pick exists. With no key, the CRM treats each row as new. Ten rows for ten known clients become ten new records.

Email is the most common key. It works well when each row has a clean email. It fails when emails are blank, shared, or typed with a slip. A team inbox used by five staff is one email for five people. A key that is wrong is worse than no key, since it can merge the wrong rows with full trust. Pick the key with care, and test it on a small file first.

Some CRMs match on name and firm. That pair sounds strong, but names change form — a topic we cover in depth later in this guide. If your CRM offers more than one key, or a strict mode and a loose mode, read its help page before the big import, not after. Five minutes of reading can save a week of cleanup.

There is a test for a weak key: run the import on a copy of the CRM, or on a test account if yours offers one, and search for five people you know are already in. If any of them shows up twice, the key is not doing its job. Fix the key, or clean the file harder — but do not shrug and run the real import. The test costs an hour. The shrug costs a season.

Cause two: your file and your CRM write the same facts in different ways

Even with a good key, form beats the match. Your CRM holds “Robert Smith” at “10 Main Street.” Your file says “Bob Smith” at “10 Main St.” A strict match sees two men, not one. The import adds Bob as new, and now Robert has a twin. This is the most common cause of import duplicates by far.

Phone numbers do the same trick. One file stores 555-010-2233. The next stores (555) 010-2233, and a third stores +1 555 010 2233. Same phone, three strings of text. A strict match treats strings, not facts. Until the forms are made the same, the match will keep missing. Cleaners call this step normalization: make each kind of value take one set form before you compare.

This is also why exact-match dedupe misses most duplicates — the match is not wrong about the text. It is blind to the fact that two texts mean one thing.

Cause three: the file itself had duplicates

Sometimes the CRM is not to blame at all. The file you fed it was already full of duplicates. Exports are a prime source. You export your list, add new rows from a fair or a form, then import the whole file back. Every old row rides along again. If the match step is weak, each one lands twice.

The guard here is simple: dedupe the file before the import, not the CRM after it. A clean file plus a fair match key will beat a dirty file plus a strict key every time. If two lists were joined to make the file, the join itself is a risk step — dedupe right after any join, while the sources are still clear in your mind.

Cause four: the import ran twice, or in the wrong order

Imports get run twice more often than teams admit. The first run seems to stall, so someone clicks again. Or two staff run the same file on the same day, each sure the other has not. If the match key works, a second run should be a no-op. If it does not, the second run doubles the harm of the first.

Order matters too. If you import the new list before you clean the old one, the match has a dirty target. Duplicates in the CRM pull in more duplicates, since a new row may match the wrong twin and leave the right one unmatched. Clean the house first. Then bring in guests.

• Name each import file with a date, and log who ran it
• Run imports one at a time, and wait for the result
• Never re-run a file until you know why the first run failed
• Import a small test batch first and check it by hand

How to stop it happening again

The fixes form one routine. First, set a match key in your CRM and write it down where your team can find it. Second, clean each file before it goes in: fix the forms of names, emails, and phones, and remove the file's own duplicates. Third, test with 50 rows. Fourth, log the import. Do this each time, and imports stop being a source of fear.

A saved cleaning routine helps most. In DedupeCSV that routine is called a recipe: your match columns, your form fixes, and your merge rules, saved once and run again in one click. On the paid plans you can keep recipes and re-run them on each new export — see the plans for the row limits. The point is not the tool. The point is that the same checks run the same way each time, no matter who is on duty.

Last, check the CRM the day after a big import, not the month after. Search five names you know. Count records before and after, and make the counts fit the file you sent. Duplicates found in a day are a small job. Duplicates found in a year have had time to collect orders, notes, and bills — and untangling them is real work.

Make one person own imports. Not a team, not “whoever has time” — a name. That person keeps the log, holds the routine, and is the one who says “stop” when a run looks wrong. Shared jobs with no owner are how files get run twice and keys get left unset. A name on the job is the cheapest fix in this whole guide, and it powers all the rest.

What a good import looks like, end to end

It helps to picture a good import once, in full. The file arrives and gets a dated name. Someone runs the pre-flight check: shape, values, dedupe. The CRM's match key is set and known. A test batch of fifty rows goes in first. A person checks ten of them by hand — five known clients, five new ones — and finds each in the right place, once.

Then the full file runs, once, while one person owns it. The counts come back: rows sent, rows added, rows matched. Those three numbers get written in a log next to the date and the file name. The next day, a spot check searches five names at random. Each appears once, with the right facts. The folder with the source file, clean file, and audit file goes on the shelf.

Notice what is missing from that story: drama. No weekend cleanup. No long thread of “why is this client in twice?” No quiet fear before the next import. A good import is a non-event. If your imports are events, one of the four causes above is still open. The story gives you a test: run it in your head against your last import, and the gaps will name themselves.

Also notice who does what. One person owns the run. The checks are written down, not held in one head. The log exists so a new hire can run the same import the same way. Imports fail in the gaps between people as often as in the gaps between systems. A short written routine closes both kinds of gap at once.

Cost is part of the picture too. The pre-flight check takes most of an hour on a big file. The cleanup after a bad import takes days — if the harm can be undone at all. Some of it cannot: a wrong merge that mixed two clients' orders may never fully untangle. An hour of care before the door beats a week of repair inside the room.

Start with your next import, not with a grand project. Pick the file that lands most often. Write its routine on one page. Run it once, by the page. Fix the page where it pinches. By the third run, the routine will feel like part of the furniture — and your CRM will stay as clean as the day you fixed it.

Already duplicated? Clean the CRM in place

If the harm is done, do not panic and do not start deleting by hand inside the CRM. Export the full list to a CSV file. Clean the file with the safe steps: strong match columns, review of each group, gap-filling merges. Then either re-import with a strict key, or use the clean file to guide hand fixes for the worst groups.

Work in waves. Start with the clear cases: exact copies and same-email groups. Then the strong near-matches. Leave the thin cases for last, or leave them in place. A few left duplicates are a small cost. A wrong merge of two real clients is a big one. Keep an audit file of each wave so any merge can be traced and, if need be, undone.

The one-line lesson

Imports do not create duplicates out of spite. They create them when the match key is weak, the forms differ, the file is dirty, or the run is doubled. Fix those four things and the same import that hurt you becomes dull and safe. Dull is what you want from an import.

If your next job is joining two lists into one file, read how to merge two mailing lists without duplicates — it covers the join step where many import duplicates are born.

One last thought: do not blame the person who ran the import. Blame the missing routine, fix it, and write it down. The next import is coming either way. It may as well be a dull one.

Sources and further reading

• Manage your business — U.S. Small Business Administration — The SBA's hub for running the day-to-day side of a small business, including keeping records and systems in order.
• Analysis Ready Datasets — UC Davis Library — UC Davis on data standardization: making values take one set form so they can be compared and joined with trust.