Loading...

Imports

Records reach a list by being imported — from a feed, from a file in one of your media stores, or from a file you upload. Every import is written down as an import run: what it was loading, how far it got, and how it ended. A run is readable while it is happening and long afterwards, which is what turns “did that load work?” from a guess into a question with an answer.

1

Every import is written down

Whenever records are loaded into a list — from a feed, from a file in one of your media stores, or from something you uploaded — the platform keeps a record of that import run: what it was loading, how far it got, and how it ended. Open a list and choose Import history from the Options menu.

A list page with the Options menu open, showing the Import history item
2

Watch a large import as it runs

While an import is in flight, the list page shows how far through the file it is, with an estimate of the time left. A big file can take a while — this is what tells you it is moving rather than stuck. The progress belongs to the import, not to your browser: close the tab, come back tomorrow, and the bar picks up wherever the import has got to.

The percentage is measured in bytes of the file read, which is why it can be shown from the first moment. A row count has no total until the whole file has been parsed — which is the slow part — so the row figure is an estimate and says so with a ~.

A list page showing an import in progress: a bar at 34% with 4.7M of ~14M rows and about 39 minutes left
3

Look back at what happened

The history lists every import into this list, newest first, with where it came from, how many rows landed and how it finished. This is the part that is worth having weeks later, when someone asks why a product disappeared: you can see whether an import ran, and what it did.

The Import history dialog listing four runs with their sources, row counts and statuses
4

Open one run for the detail

Pick a run to see its counters: rows read from the file, rows imported, and rows that could not be used. A healthy import usually loses a few rows — a blank name here and there — and the run keeps a sample of where, with line numbers, so you can open your file at the right place instead of hunting.

One import run: succeeded, 13,981,204 imported of 14,002,331 read, with the first affected line numbers
5

A file that is going nowhere is stopped early

If a file starts losing most of its rows — the fields do not match the list, or a gate is refusing the records — the import is stopped in the first few thousand rows rather than spending an hour proving it. The run reads Stopped early, in amber rather than red: nothing broke, the platform read enough to judge the file and chose not to grind through the rest.

Rows imported before that point are kept. Stopping is not a rollback. Fix the mapping or the gate and run the import again — re-importing the same records is harmless.

An import run marked Stopped early, explaining that 92% of rows could not be imported and that earlier rows were kept
6

Feeds link to their last import

A feed's job is to keep its file current; making that file searchable is the import's job. They can disagree — a feed can sync perfectly into a file the import then refuses. The feed row's ⋮ menu has Last import, which opens the run that consumed its file, so “the feed says synced but the records look wrong” is one click to answer.

Automatic imports also speak up now. If one fails for a reason that will not fix itself — a file that cannot be parsed, a file over the size limit, a run stopped early — the account owner is emailed. Once per problem, not once per retry: a broken file that is retried every fifteen minutes does not email you every fifteen minutes.

The Feeds page with a row menu open, showing the Last import item
Where to find it

Open a list and choose Import history from the Options menu. Every import into that list is there, newest first, with its source, row count and status. Pick one to see its counters and anything it could not use.

While an import is running, the list page carries a progress bar above the records. It reads from the run itself, not from your browser — closing the tab loses nothing, and reopening the page shows wherever the import has got to.

What a run tells you
FieldMeans
SourceWhich feed, media-store file or upload the records came from.
StatusRunning, Succeeded, Failed, or Stopped early (see below).
RowsRead from the file, imported into the list, and skipped or refused.
FileThe file’s size, when the store reported one. “Unknown” when it did not — the progress bar then shows a row count and no percentage, rather than inventing one.
First affected linesA sample of the rows that could not be imported, with line numbers, so you can open your file at the right place.
Progress, and the estimate

Percentage is measured in bytes of the file read. That is the only measure available from the first moment: a row total is unknown until the whole file has been parsed, which is the slow part. The row figure beside it is extrapolated from how much of the file has been read, which is why it carries a ~.

The time remaining appears once the import is 2% in. Before that the sample is too short to divide by, and an estimate that swung from four days to four minutes in the first seconds would be worse than none.

A list page showing an import in progress: a bar at 34%, with 4.7M of ~14M rows and about 39 minutes left
When an import is stopped early

A file whose fields do not match the list loses nearly every row, and it loses them from the first row to the last. Rather than spend an hour proving that, the platform judges the file once a few thousand rows have been read: if more than a quarter of them could not be imported, the run is stopped early and says why.

Two things worth knowing:

  • Rows imported before the stop are kept. Stopping is not a rollback. Fix the mapping (or the gate that is refusing the records) and run the import again — re-importing the same records is harmless.
  • It counts rows that could not be used — rows with no name, and rows a gate refused or held back. A row that imported with one long value truncated is a warning, not a loss, and never counts towards the limit.
An import run marked Stopped early, explaining that 92% of rows could not be imported and that earlier rows were kept
Automatic imports now speak up

A list bound to a file with keep-in-sync re-imports whenever that file changes, quietly and on its own. That silence is right for a passing problem — a store briefly unreachable, a file caught mid-write — because it fixes itself and an alert would be noise.

It is wrong for a problem that will never fix itself. A file that cannot be parsed, a file over the size limit, or a run stopped early now emails the account owner and shows in the console — once per problem, not once per retry. A broken file retried every fifteen minutes does not email you every fifteen minutes; fix it, and the next failure (a different one) is news again.

Feeds and imports are two different jobs

A feed keeps its file current. An import makes that file searchable. They can disagree — a feed can sync perfectly into a file the import then refuses — which is why the feed row’s menu has Last import: it opens the run that consumed that feed’s file. If a feed says Synced but the records look wrong, that is the first place to look.

The Feeds page with a row menu open, showing the Last import item
Snapshot files: what is not in the file is removed

A file can declare itself a snapshot — the complete truth about a list, not a set of changes. When it does, records the file no longer contains are retired from the list after the import: absence means the source deleted them.

This used to be limited by size, because working out what was missing meant holding the whole file in memory. It no longer is: each record now remembers which import last wrote it, so the platform can ask for “everything this run did not write” directly. A snapshot of any size reconciles the same way.

An update-only file (no snapshot declaration) never removes anything — records it does not mention are simply left alone.

Read imports from the API

Import runs are readable from the console today. An API surface for polling a run is deliberately held back until the run carries everything an automated caller would want from it; if you need to drive imports programmatically now, start the import through the existing import-from-media endpoint and watch the list’s record count.

Top