↑↓ selectEnter openEsc close

    1. Elbuz

    What to do when the site parser collected the wrong data or nothing

    5 min readUpdated

    Elbuz logs every crawled page in the “Parsing log” with its response code and size: was the page not served, or do selectors no longer parse it?

    When you need this:

    • Run done, but no products or empty names — “Operation type” shows where the chain broke off.
    • “I'm not a robot” check — collection pauses until you pass it in your browser.
    • 403 or 429 responses — the page is closed after three failed attempts; the refusal page stays out of the price list.
    • Site redesigned — rule test, selector wizard, and AI matching restore working rules.

    Where to start: open the “Parsing log” before the next run.

    What helps you find the cause

    Which log answers your question

    The most common confusion in this module isn't about selectors but about logs. There are three of them, and they answer different questions.

    The live feed in the “Import data from the site” window answers “what is happening right now”. Rows arrive as the work goes: “Starting download”, “Loading data from links - N (M)”, where N is the pages in this batch and M the jobs left, “Captcha detected”, “Operation canceled by the user”, “Apply different operations to products”, “Job completed”. It isn't an archive: rows arrive in real time and are never replayed, so a reopened window is empty, and the previous run's entries are wiped once you start the next one.

    “Parsing log”: one row per page

    This window answers “what happened to each page”. It is essentially a job queue: one row is one page that had to be opened and parsed. Open it with the “Parsing log” button at the top of the “Additional” tab or with the log icon in the collection window; it is already filtered by your source.

    If you open the log between runs and see “No data to display”, that's normal, not a setup failure. The queue lives only as long as the crawl runs: at the start of each run the app clears it, along with the detailed rows of the general log for this source. So check the log during the run or right after it; for history over past days, use the “Log” tab in the source card.

    Columns, in order of usefulness: “Operation type” (category links, product links, product card, product attributes), “Link”, “HTTP response code”, “Size, KB”, “Retries”, “Status” — a cumulative field where the app appends reasons, “Date added”, “Done”, “In progress”, “Response time”, “Completed on”, “Contact / Price list”, “User”, “Job ID”. The page icon in a row opens “HTML page code”, but only if “Debug mode” was on in the settings; otherwise the markup isn't saved.

    The “Operation type” column is the most useful and the least obvious: it shows which pipeline step the crawl reached. If the log has “Category links” and “Product links” rows but no “Product card (content)”, the chain broke off at the second step — the product block selector found nothing. If there are cards but no attributes, the problem is in the fourth step.

    The toolbar has “Refresh” and “Stop parsing data” — the second button stops collection without opening the run window.

    The source run log answers “how many rows arrived in the end”. It is the same “Log” tab in the source card as for file price lists, with a health score, tiles, and history; the “Format” column shows parser for such a source.

    The site changed its look and the wrong data was collected

    This is the most common collection failure, and it looks harmless: the run completes, there are no errors, and no products either. Or there are products, but with empty names. The cause is almost always the same: the site was redesigned, and the selectors no longer find what they found last year. You fix it in four steps, and the order matters.

    Step one — read the “Parsing log” (button at the top of the “Additional” tab): it tells “the site didn't serve the page” apart from “the page arrived but wasn't parsed”, and the “Operation type” column shows where everything stopped. Do it before the next run: “Start download” wipes both the job queue and the live feed of the previous run, leaving nothing to look at. “Continue” wipes nothing. The signs:

    • response code 200, normal “Size, KB”, but no products — the page arrived intact, the selectors are the problem;
    • code 403, 429, or 5xx and a “Status” with http … — the page did not open, skipped — the site refused, the selectors aren't to blame, check the fetch mode;
    • code 404 and “Done” — the page no longer exists, remove the address;
    • a suspiciously small “Size, KB” — instead of the page, an anti-bot stub or an “I'm not a robot” check arrived;
    • no jobs at all, and the feed immediately shows “Job completed” — there's nothing to crawl: either “Links” is empty along with the category selector, or the “Operation selector” found no links on the page.

    Step two — test the rule at the step where the chain broke off. Enter the address of a page that definitely exists in the “Test Link” column and click the test button in the “Operation selector” row. The left “Test result” panel shows what the selector returns now. Empty — the selector is dead; something, but not the right thing — the selector catches a neighboring block.

    Step three — pick the selector again. For list operations, use the “Parser configuration wizard”: it shows ready selectors for all link groups on the page, and you just copy the one you need into “Operation selector”. For the product card and attributes, use the AI matching button, which rewrites the rules by itself.

    Step four — turn on “Auto-detect selectors” if the site is redesigned regularly. In this mode, on every run the app picks a selector by itself wherever it is empty, and tells a list page from a card by typical listing words and price marks.

    When the wrong data came from the site

    When the site doesn't serve pages

    Sites protect themselves from automated crawling, and that's a normal part of the job. The app distinguishes two cases.

    An “I'm not a robot” check. This case has its own behavior: collection pauses, the page goes back to the queue, a sound plays, and the feed shows “Captcha detected” in red with the link “Pass the check on the site and press “Continue””. The link leads to the exact page where the check appeared and opens in a regular browser tab with your cookies: pass the check, click “Continue”, and the crawl resumes from the same place.

    A Cloudflare “Just a moment…” check. The app recognizes it in any mode, even without your own captcha signs, and puts collection on the same pause. In browser mode, the extension (version 2.4.3 and later) first waits up to 12 seconds for the check to pass by itself. If it doesn't, the tab with the check opens in the parsing window: pass the check there, and collection resumes by itself.

    The app recognizes a check by its signs — typical pieces of page code stored in the parser's own settings. Ready-made modules have them filled in: for Hotline, these are g-recaptcha, “Automatic checking against unwanted bots”, and a few more lines. The universal parser created by the wizard has no signs at all — the key thing to know: a page with a check reaches it as a normal one. In the parsing log it has code 200, a suspiciously small size, and zero products. Seeing this, open that address in your browser and check what's really there.

    A server refusal. Codes 400+ mean the page didn't open. A 404 is closed at once for good — the page is gone. Others (403, 429, 5xx) are retried; after the third failed attempt the job is closed. The response body is never saved as a parsing result: an “Access Denied” page won't become a product description. To your “Ignore pages containing text” lines, the app adds ten service signs — Cloudflare stubs and checks, “403 Forbidden”, “Access Denied”, etc. Server mode only: in browser mode the extension brings the page, with no such safety net.

    What to do depends on the site. The supplier gave you access — “Directly, without a proxy”: such requests aren't charged. The site just dislikes strangers — “Through a proxy (regular server mode)” or your own addresses in the “List of proxy servers” window (one per line, as user:password@proxyserver:proxyport or proxyserver:proxyport). Neither helps — “Through a protection-bypass service”: it gets past CAPTCHAs and anti-bot protection of large marketplaces, but each request is charged separately. When the balance runs out, the run stops with “Premium parsing (anti-bot bypass) stopped: insufficient balance. Top up your balance to continue.”

    Scenarios

    The night run finished without errors, but there are zero products

    Open the “Parsing log”. Code 200 and a suspiciously small “Size, KB” mean that an “I'm not a robot” check or a protection stub arrived instead of the page: the universal doesn't recognize it. Open the address in your browser and change “How the server fetches this site” (about blocking).

    What won't happen: the stub page won't get into the price list as a product — it simply yields zero products.

    After a site update, names arrived empty

    The log shows code 200 and a normal size — the page is intact, the selectors are outdated. Run the rule test at the step where the chain broke off and pick the selector again: with the wizard for lists, with AI for the card and attributes (order of checks).

    What won't happen: AI matching doesn't keep the old values — copy the working selectors before running it.

    The site started responding with 403

    If the supplier gave you access, set “Directly, without a proxy”. If not, use “Through a proxy (regular server mode)” or your own addresses in the “List of proxy servers”, and if that doesn't help either, “Through a protection-bypass service”.

    What won't happen: refused pages aren't retried endlessly — after the third failed attempt the job is closed.

    Parser messages and what to do about them

    What you seeWhyWhat to do
    The parsing log shows code 200, “Size, KB” is noticeably smaller than usual, and these pages yield no products What arrived wasn't the page but a protection stub or a Open the same address in your browser. If there's a check, pass it or change the page fetch mode (about blocking)
    Response code 200, normal page size, but no products The page arrived intact, but the selectors no longer parse it — the site changed its look Rule test → selector wizard → AI matching, in exactly this order. In server mode, also check the “Ignore pages containing text” field (the site changed its look)
    In the parsing log: http 403 — the page did not open, skipped The site refused. The response body isn't saved as a result; after the third failed attempt the job is closed Change “How the server fetches this site” or connect your own proxies. If the supplier gave you access, use “Directly, without a proxy” (about blocking)
    The same with code 404 The page no longer exists; the job is closed at once, without retries Remove the address from “Links” or reload the sitemap
    In the log: op_status = 1 no response for more than 35 sec, attempt 2 / 5 The response didn't arrive in time. In browser mode (35 seconds), the window was closed, the browser minimized, or the extension is missing; in server mode (150 seconds), the site is slow Don't close the window during collection, or turn on “Use server to load data” (about the two modes)
    The last log row for a page: op_status = 1 no response for more than 35 sec, attempt 5 / 5 — the second number equals the first Retries after timeouts are used up: the page was closed without data and skipped. If retries on empty responses are used up, the row is different: op_status = 2 failed to load after 5 attempts Open the address from the “Link” column in your browser. It opens — repeat the run; it doesn't — the site blocks requests (when the site doesn't serve pages)
    After a content search: “Search finished, but 5 sources returned errors” In a browser run, five source sites responded with an error or a CAPTCHA at least once. Below the heading is a breakdown: “site name — N failures · captcha M” Click “Open the log” — the “Progress” tab opens with the “Errors only” filter. CAPTCHA and 403 — the site blocks; code 200 with nothing found — the site changed its layout
    In the log: op_status = 3 received an empty response The page came back empty. The job returns to the queue and is retried until the five attempts are used up Check the address in a browser. If the page opens but empty responses keep coming, turn on “Use server for rendering JavaScript”
    “fetch service returned no HTML, skipped” The external page fetch service returned an empty response Repeat the run; if it happens again, change the page fetch mode
    “Premium parsing (anti-bot bypass) stopped: insufficient balance. Top up your balance to continue.” “Through a protection-bypass service” is selected, and the balance has run out Top up the balance or switch back to “Through a proxy (regular server mode)”

    FAQ

    I opened the parsing log and it says “No data to display”. Is something broken?

    No. The queue lives while the crawl runs: at the start of each run the app clears it. For history over past days, use the “Log” tab in the source card (about the log).

    Can I export the “Parsing log” to Excel to send it to the supplier?

    No, there's no export. Open “HTML page code” in a row — it's what the site served to the app (only with “Debug mode”) — or select and copy rows from the table. Do this before the next run of the same source.

    Where can I see the HTML of the page the app fetched?

    In the “Parsing log”, via the page icon in a row — “HTML page code” opens. The markup is saved only when “Debug mode” is on in the “Additional” tab.

    The log says “no response for more than 35 sec, attempt 2 / 5”. Is that bad, is the product lost?

    No, it's a retry after a timeout: the job went back to the queue. It's only bad when both numbers are the same — “attempt 5 / 5”: the page was closed without data (what to do).

    Will an “Access Denied” page get into the price list as a product description?

    No. A response body with code 400 or above is never saved as a parsing result.

    Where to find it and who has access

    Menu
    Directories → Processing price lists → List of price lists
    Parsing log
    parser settings window → “Additional” tab → “Parsing log”
    Other entry points
    the log icon in the collection window; “Additional operations” in the price list products window
    Access right
    the “Parsing log” or “Website parsing log” window
    Prepare
    “Debug mode” on the “Additional” tab — so the log keeps the page code

    Related topics

    Was this article helpful?

    Missing a feature? Tell us.

    We will add it for free if other users need it too.

    If only you need it, we will agree on it separately.

    Article contents

    Suggest an improvement

    Try Elbuz free