Most businesses analyse their Google Search Console data by clicking the export button. There is a problem with that approach: the reports usually show only 1,000 rows, and the interface keeps just 16 months of performance history.
Google’s bulk data export removes both limitations. It sends your Search Console performance data to Google BigQuery every day, without the usual row limit. Anonymised queries are excluded, but the remaining data can be stored for as long as you need it.
This opens up new opportunities for SEO analysis. You can analyse much larger query sets, compare performance over several years, build your own dashboards and connect search data with sales, CRM and analytics data.
However, setting up the export is only the beginning. If the pipeline stops working, data can be lost permanently. Poorly written queries can also produce incorrect figures or run up unexpected BigQuery bills.
In this guide, we explain how bulk data export works, what it costs, how to set it up properly and how SEO, marketing, content and development teams can use it.
1. What Happened?
Four Ways to Export Google Search Console Data
Google offers several ways to access Search Console performance data. In its Search Console Training series, Search Advocate Daniel Waisberg compares the available options.
The main differences are the amount of data you can retrieve, how much automation is possible and how useful each method is for long-term analysis.
| Export method | Data limit | What it covers | Best use |
|---|---|---|---|
| Export button in Search Console | Usually 1,000 rows of representative examples | The report currently on screen | Quick checks and smaller websites |
| Looker Studio connector | Up to 50,000 rows | Performance data | Dashboards and regular reporting |
| Search Console API | Up to 50,000 rows per day for each search type, retrieved in batches of up to 25,000 | Performance data, with other API methods available for site information, sitemaps and URL inspection | Automated reporting and historical data collection |
| Bulk data export to BigQuery | No standard daily row limit; anonymised queries are excluded | Search Console performance data | Large websites, detailed analysis and long-term reporting |
Google designed bulk data export particularly for large websites with tens of thousands of pages, or websites receiving traffic from tens of thousands of queries each day.
However, you do not need to run a huge website to benefit from it. If your Search Console reports regularly reach their row limits, you could already be missing useful detail about your search performance.
How Bulk Data Export Works
Bulk data export sends your Search Console performance data to a dataset in your own Google Cloud project. The export runs daily, creating a long-term record of your search performance.
There are several important things to understand before enabling it.
- It does not backfill historical data. The first export normally arrives within 48 hours of setup. Data from before activation must be collected separately through the Search Console API or existing reports.
- Your data can accumulate indefinitely. BigQuery retains the exported tables and their partitions by default, unless you configure an expiry period or your organisation applies a different retention policy.
- You pay for storage and queries. BigQuery has a free usage allowance, but charges may apply when your storage or query activity exceeds it.
- A property owner must enable or disable the export. Search Console also notifies property owners and full users when export errors occur.
- Each property needs its own dataset. Two Search Console properties cannot export to the same dataset.
- Choose the dataset location carefully. Moving the export to another location is not a straightforward setting change. It involves stopping the export, copying the data and setting it up again.
The biggest advantage is that you can build your own reporting system around the data. The biggest responsibility is making sure the export continues to work.
The Three Tables Created by Bulk Data Export
Search Console creates three tables in your BigQuery dataset.
| Table | What it contains | What to use it for |
|---|---|---|
searchdata_site_impression | Performance data grouped by property, date, query, country, search type and device | Property-level totals and query analysis |
searchdata_url_impression | Performance data grouped by URL, with additional search appearance fields | Page-level analysis and individual search features |
ExportLog | A record of successful exports for each table and date, including a version number when Google updates previously exported data | Monitoring missing data and identifying revisions |
The first two tables are the main source of your reporting data. The third helps you check whether the export is complete and whether Google has updated historical data.
Two details in Google’s documentation catch out many first-time users.
First, rows can repeat. Search Console exports data incrementally, so the same combination of query, date and search type can appear in several rows. You should normally aggregate the metrics using SQL functions such as SUM() before reporting them.
Second, average position must be calculated correctly. The stored position values are sums based on zero-indexed positions, where the top position is zero.
For the site-level table, calculate average position using:
SUM(sum_top_position) / SUM(impressions) + 1For the URL-level table, use:
SUM(sum_position) / SUM(impressions) + 1Do not simply average the stored position values. The result will be wrong.
What Can Go Wrong With the Export?
Google’s documentation explains how the export behaves when something goes wrong.
- Temporary problems: Short-lived issues, such as connection errors, are retried automatically.
- Missed export days: Problems such as missing permissions, invalid billing arrangements or exceeded quotas can stop an export. Google retries missed data for about a week before giving up on that date.
- The export eventually stops: After about a month of continued export failures, Search Console stops the bulk export entirely.
- Failed exports are not recorded in
ExportLog. You need to check the latest export status in Search Console and investigate errors in Google Cloud Logs Explorer. - Changes to the export table schema can break the pipeline. Do not add your own columns to Google’s export tables.
A dashboard can continue displaying old or incomplete data even after the export has failed. This is why monitoring is essential.
Search Console Data Anomalies to Account for in 2026
Google has documented several data problems that affect long-term Search Console reporting. These are particularly important if you use BigQuery to compare performance over time.
| Date | What happened | What it means for your data |
|---|---|---|
| 28 February and 1 March 2026 | Some properties are missing these dates from their bulk exports. Google says the data will not be recovered. | Check your ExportLog table for these dates. If the data is missing, record the gap in your reporting. |
| 13 May 2025 to 27 April 2026 | A logging error affected reported impressions. CTR and average position were also affected, but clicks were not. Google resolved the issue in April 2026. | Year-on-year impression, CTR and position comparisons covering this period need a caveat. |
| 16–27 April 2026 | A logging error affected impressions and clicks for the Job listing and Job details search appearance types. | Recruitment websites should account for this gap in their historical analysis. |
| From 7 May 2026 | FAQ rich results stopped appearing in Google Search. | The is_tpf_faq field no longer provides useful data for recent dates and should not be relied upon for current FAQ performance reporting. |
| Several dates between May and August 2026 | Logging errors affected Discover and generative AI features in Discover. | Annotate the affected dates when analysing Discover trends. |
| 13–17 August 2026 | A logging error reduced reported impressions for generative AI in Search. Google confirmed on 21 August that the missing data had been restored in the Performance report. | Check ExportLog for revised versions of these dates and rebuild any summaries that rely on older exported data. |
These issues are a reminder that a complete export does not necessarily mean every figure is accurate.
You need to understand both the health of your own data pipeline and any reporting problems Google has identified.
2. Why Does Bulk Data Export Matter?
Bulk data export gives you a much more complete record of your website’s search performance. However, you only benefit from it if the export runs reliably, the costs remain under control and your queries calculate the metrics correctly.
There are five main reasons to take it seriously.
1. Every Month Without an Export Means Lost Historical Detail
Search Console retains 16 months of performance data in its interface, and enabling bulk export does not recover the history from before activation.
For example, a business that enables the export today will start building a more complete record from that point onwards. It will still need to use the API or existing reports to collect the historical data that remains available.
This matters when you need to:
- Compare search performance year on year.
- Measure the impact of a website migration.
- Evaluate the results of a large content programme.
- Track changes in query demand over several years.
- Analyse how customers discover and reach important pages.
It also matters for advanced SEO research. Detailed query data that can be difficult to obtain reliably from the standard Search Console interface.
The longer you wait, the more historical detail you risk losing permanently.
2. Data Can Disappear Without Anyone Noticing
Bulk export failures often result from ordinary IT or administrative changes.
For example:
- A billing method expires or the billing account is disabled.
- A Google Cloud project is moved or reconfigured.
- A security review removes an account that the export depends on.
- Someone changes the export table’s schema.
- A quota is exceeded or access permissions are removed.
Google retries missed exports for about a week. After that, the data for the affected date is no longer available through the bulk export. After about a month of continued failures, the export stops altogether.
The problem is that failed exports do not appear in ExportLog. A reporting dashboard may continue to display old data without an obvious warning.
You therefore need automated checks that confirm the latest dates are present and that both performance tables have been exported.
3. BigQuery Costs Can Grow Quickly
Storing Search Console data is relatively inexpensive. Repeatedly scanning large tables can be much more expensive.
At the current published US list prices for BigQuery, active logical storage costs approximately $0.023 per GiB each month, with the first 10 GiB free. Data in partitions that have not been modified for 90 days qualifies for the lower long-term storage rate of approximately $0.016 per GiB each month.
For on-demand queries, the first 1 TiB processed each month is free, followed by a published rate of $6.25 per TiB in Iowa (us-central1). Actual charges depend on your region, usage and pricing arrangement.
BigQuery charges for the data processed in the columns you select. Adding LIMIT to a query does not reduce the amount scanned.
That means a dashboard which scans a large URL table every time it refreshes can cost more than storing the underlying data for years.
The developer toolkit later in this article includes cost estimates and practical settings to help prevent unexpected bills.
4. Incorrect Queries Produce Convincing but Wrong Reports
Many errors happen because analysts treat the exported tables as if they were already clean, consolidated reports.
Here are the mistakes to avoid.
| Mistake | What happens |
|---|---|
| Reading individual rows without aggregation | Metrics can be undercounted because the same reporting keys can appear across multiple rows. |
| Averaging the stored position values | Average position is calculated incorrectly because the stored values are zero-based sums. |
| Adding impressions from the URL table to calculate site totals | Metrics can be overcounted because a search can show several URLs from the same website. Use the site table for property-level totals. |
| Using UTC dates in reporting | Reporting dates can differ from Search Console, which uses Pacific Time. |
| Ignoring anonymised queries | Query-level totals may omit a substantial amount of traffic if anonymised queries are not accounted for separately. |
| Comparing data across known anomaly periods without annotation | Historical trends can be misleading because of Google’s logging errors. |
| Relying on deprecated search appearance fields | Reports can show no data for features that no longer appear in Google Search. |
Even Google’s example queries should be checked before being used in production. Copy them as starting points, not finished reports.
5. Search Data Becomes Part of Your Wider Business Reporting
Once your Search Console data sits in BigQuery, you can connect it with other business datasets.
This creates opportunities to compare search demand with sales, leads, CRM records and analytics data.
For example, you can investigate which query groups attract visitors who eventually become customers, which categories generate the most organic revenue, and whether changes in search demand precede changes in sales.
However, that also means your Search Console dataset needs proper ownership, access controls, cost controls and a retention policy.
| Risk | Cause | Consequence |
|---|---|---|
| Lost historical data | Bulk export was never enabled | Missing query and page detail that cannot be recovered through the export |
| Silent data gaps | Billing, permissions or schema problems | Missing days followed by a possible shutdown of the export |
| Rising costs | Queries scan too much data; no retention policy is in place | Unnecessary storage and query charges |
| Incorrect reporting | Poor aggregation, wrong position calculations or mixing the two performance tables | Numbers that disagree with Search Console |
| Uncontrolled access | No clear data owner or access policy | Unnecessary security and compliance risks |
3. Who Should Use Bulk Data Export?
Any website whose Search Console data exceeds the standard report limits should consider bulk data export. The benefits become greater as the number of URLs, queries, markets and reporting requirements increases.
E-commerce Websites and Marketplaces
E-commerce websites can have thousands of product, category and filtered URLs. Each URL may attract a long list of search queries.
The standard interface cannot provide the same depth of analysis across such a large catalogue.
The URL table also includes search appearance fields, which can help with analysis of product snippets, merchant listings and other supported search features.
Publishers and News Websites
Publishers often need to analyse Google Search, Discover and Google News performance across large numbers of articles.
Their query sets can change quickly, and the ability to keep a longer history makes it easier to identify which topics and article types continue to attract visitors.
The logging problems affecting Discover in 2026 also demonstrate the value of annotating historical data before drawing conclusions from it.
Enterprise and Multi-Market Brands
Large businesses often manage several websites, countries, languages and SEO teams.
Because each Search Console property needs its own export dataset, businesses should plan how those datasets will feed into a central reporting environment.
Once the data is available in one place, teams can compare markets, analyse shared query themes and connect organic search performance with commercial results.
SaaS, Finance and Lead-Generation Businesses
In businesses with longer sales cycles, the search query that first introduces a customer to the company may be far removed from the query that generates the eventual conversion.
Fuller query data, combined with CRM outcomes, can help uncover which search themes contribute to lead generation and customer acquisition.
Businesses Planning a Website Migration or Major Content Change
A migration or significant content programme needs a reliable baseline.
You should enable bulk export before the change, where possible, so you have a more complete record of query and page performance.
Enabling it after launch means you cannot recreate the missing bulk-export history.
Smaller Websites
Bulk export is not essential for every business.
A small website with only a few hundred queries each month may be able to work comfortably with Search Console’s interface or occasional API exports.
In these cases, BigQuery may introduce more setup, costs and maintenance than the business needs.
The decision should be based on how much detail you need, the scale of the data and what you intend to do with it.
4. What Should Businesses Do?
Treat bulk export as a data pipeline that needs an owner and regular checks, not as a setting that someone enables once and forgets.
The following process covers the important decisions, initial setup, historical data collection, cost control, query writing, monitoring and rollout.
Step 1: Make Four Decisions Before Enabling the Export
Some setup decisions are difficult or expensive to change later. Agree on them before anyone starts configuring the export.
| Decision | Who should decide? | Recommendation | Why it matters |
|---|---|---|---|
| Google Cloud project and billing | Data or IT lead, working with finance | Use the project that already stores marketing and sales data if you plan to join the datasets. Otherwise, use a dedicated business-owned project with its own budget. | Billing failures can interrupt exports. |
| Dataset location | Data lead | Use the same location as the datasets you intend to join to. | BigQuery cannot join datasets in different locations in the same query. Moving the export later requires extra work and may leave a gap. |
| Search Console properties | SEO lead | Select the appropriate domain property and any URL-prefix properties needed for separate markets or sections. | Each property needs its own export dataset. Site-level totals should come from the site table, not by adding up URL-level data. |
| Data retention | SEO lead and data owner | Keep at least 16 months if that is sufficient for your reporting, or around 25 months or longer for year-on-year analysis. | Expired partitions are eventually deleted, so retention needs to match your reporting requirements. |
Step 2: Set Up the Export
The first part of setup happens in Google Cloud. The second happens in Search Console.
In Google Cloud
1. Choose the project and confirm billing.
Choose an existing Google Cloud project or create a new one. Confirm that it has an active billing account.
Record the project ID. This is the ID you will need when configuring the Search Console export.
2. Enable the required APIs.
Under APIs and Services, enable the BigQuery API and BigQuery Storage API as required for your planned workflow.
3. Configure the service account’s permissions.
In IAM, grant the Search Console export principal:
search-console-data-export@system.gserviceaccount.com
the following roles:
- BigQuery Job User.
- BigQuery Data Editor.
Follow Google’s current setup instructions and your organisation’s access policies when granting permissions.
4. Check organisation-level restrictions.
Some organisations restrict which accounts can access Google Cloud resources. If a domain restriction policy prevents setup, involve your Cloud administrator before proceeding.
Google documents a specific setup process for projects affected by these restrictions.
In Google Search Console
5. Open the bulk data export settings.
Open the relevant Search Console property, then navigate to Settings and Bulk data export.
6. Enter the Cloud project details.
Provide the project ID, dataset name and dataset location.
The default dataset name begins with searchconsole. You can use a suffix, such as searchconsole_uk, to make the dataset easier to identify. Each property still needs its own dataset.
7. Confirm the setup.
Once the export is activated, Search Console notifies property owners and full users.
8. Check that the first export arrives.
The first export normally arrives within 48 hours.
Check the export status in Search Console and verify the results using the health-check query in the developer toolkit below.
During the First Week
Ask the developer or data team to complete these additional tasks:
- Set up the agreed partition-retention settings and query-cost controls.
- Schedule the export alert query from the developer toolkit, with email notifications enabled.
- Document the Search Console export service account in your IAM records so it is not accidentally removed during a security review.
- Create a separate reporting dataset, such as
seo_reporting, in the same location as the export dataset.
Build your own reporting tables in that separate dataset. Keep Google’s original export tables unchanged, apart from permitted settings such as partition expiration.
Step 3: Collect the Historical Data Still Available
Bulk export starts from the activation period. To collect older data, use the Search Analytics API, which can access the rolling 16-month historical window available through Search Console.
Do this as soon as possible after setting up the export. Every day you wait, another day moves towards the edge of that historical window.
For each date and search type you need, collect two types of data.
First, collect daily totals.
Request data grouped by date, without query or page dimensions. These totals are useful for historical reporting and comparison with the Performance report chart.
Second, collect detailed query and page data.
Request query and page dimensions, adding country and device if required.
The API can return up to 50,000 rows per day per search type. That is a limit, not a guarantee that all possible query and page combinations will be returned. Search Console may return only the leading rows.
This means the API is useful for collecting historical detail, but it cannot replace the bulk export for a full long-term record on a very large website.
How to Request the Data
- Request one day at a time, using
rowLimitof 25,000. - Increase
startRowby 25,000 to retrieve subsequent batches. - Stop when the API returns fewer rows than requested. If you receive exactly 25,000 rows, make another request to check whether more are available.
- Use
dataState: "final"to collect only finalised data. - Give the account running the backfill the read-only scope,
webmasters.readonly, if it only needs to read data.
Write the results to your reporting dataset, for example seo_reporting.api_backfill, and record that the source is the API.
Never write API backfill data directly into Google’s export tables. These tables have a schema managed by Search Console. Adding columns can break the export, and API data may be incomplete compared with bulk-export data.
Keep the two sources separate so that analysts can distinguish the completeness and limitations of each.
Step 4: Put Cost Controls in Place From Day One
Storage costs are generally manageable. Unrestricted queries and poorly designed dashboards are a much bigger risk.
Use the following controls before connecting the dataset to reporting tools.
1. Set Partition Expiration, Not Table Expiration
Partition expiration limits how long individual date partitions are retained.
For example, if you need approximately 25 months of history, you could use a retention period of 761 days, subject to your reporting and compliance requirements.
Google recommends avoiding very short expiry periods for bulk export because the export needs enough time to write and retry data. Its table guidance recommends partition expiration of 14 days or more.
Be careful when changing the setting: partitions older than the new limit become eligible for deletion, and a table-level expiry can delete the entire table.
For example:
ALTER TABLE `acme-seo.searchconsole.searchdata_site_impression`
SET OPTIONS (partition_expiration_days = 761);
ALTER TABLE `acme-seo.searchconsole.searchdata_url_impression`
SET OPTIONS (partition_expiration_days = 761);Replace the project, dataset and table names with your own.
Before applying a retention policy, check that you will not delete data needed for longer-term analysis.
2. Set a Maximum Query Cost
Use BigQuery’s maximum bytes billed setting to prevent individual queries from scanning more data than you have approved.
You can configure this in the query settings or pass --maximum_bytes_billed when using the bq command-line tool.
If a query would exceed the limit, it fails rather than running and generating a larger bill.
3. Set Daily Query Limits
Use project-level and user-level query quotas where appropriate.
These controls help prevent an incorrectly configured dashboard or script from scanning large datasets repeatedly throughout the month.
4. Set a Billing Budget Alert
Create a budget alert in Google Cloud Billing.
Choose a threshold based on your expected storage and query costs, leaving enough room to investigate unexpected usage before it becomes expensive.
Remember that budget alerts notify you; they do not automatically prevent all spending.
5. Connect Dashboards to Summary Tables
Avoid pointing Looker Studio and other reporting tools directly at the raw export tables for routine reporting.
Instead, build smaller summary tables that contain only the dimensions and metrics your dashboards require.
For example, the page_daily table in the toolkit below stores daily page-level metrics. This reduces the amount of data that needs to be scanned repeatedly.
6. Write Queries That Scan Less Data
Follow these rules:
- Select only the columns you need.
SELECT *can increase the amount of data processed. - Filter by
data_dateso BigQuery can skip date partitions that are not relevant. - Use fixed date expressions where possible. A partition filter that depends on a subquery can prevent partition pruning in cases where BigQuery cannot determine the relevant partitions in advance.
- Check the estimated bytes processed before running expensive queries.
- Remember that
LIMITrestricts the number of rows returned, not the amount of data scanned.
What These Controls Can Mean for Cost
Consider a large website exporting two million site-table rows and five million URL-table rows each day.
The following estimates illustrate how storage and query costs can differ. They use the article’s example assumptions and published Iowa list prices, rather than a quote for your own project.
| Cost item | Example estimate |
|---|---|
| New data each month | 34.5 GiB |
| Stored data after 25 months | 863 GiB |
| Approximate monthly storage charge after allowances | $14 |
| Dashboard scanning 180 days of the raw URL table, refreshed 24 times daily | About $442 per month |
| Similar reporting based on a daily summary table | About $39 per month |
These figures are illustrative. Your actual costs depend on the number and size of rows, selected columns, query filters, refresh frequency, region and pricing arrangement.
Measure your actual row counts and use BigQuery’s dry-run estimates and billing reports before deciding on a budget.
Step 5: Follow the Same Rules When Writing Every Query
Most reporting errors happen when different analysts write queries in different ways.
Set clear rules for the whole team.
| Rule | How to apply it | Why it matters |
|---|---|---|
| Aggregate metrics | Use SUM(clicks) and SUM(impressions), grouped by the dimensions you need. | Data may be spread across multiple rows with the same keys. |
| Calculate average position from sums | Site table: SUM(sum_top_position) / SUM(impressions) + 1. URL table: SUM(sum_position) / SUM(impressions) + 1. | Stored positions are zero-based sums, not ready-to-average values. |
| Choose the correct table | Use the site table for property-level totals and the URL table for pages and search appearance analysis. | Summing URL-level impressions can overcount searches where several of your URLs appear. |
| Use Pacific Time for dates | For example, CURRENT_DATE('America/Los_Angeles'). | Search Console’s reporting dates use Pacific Time, not UTC. |
| Match search-type values correctly | For example, UPPER(search_type) = 'WEB'. | Google stores lower-case search-type values, although its sample queries sometimes use upper case. |
| Account for anonymised queries | Check is_anonymized_query and the query field, including empty strings and nulls. | Anonymised queries are real traffic, even when the query itself is hidden. |
| Filter dates using partition columns | For example, data_date >= DATE_SUB(CURRENT_DATE('America/Los_Angeles'), INTERVAL 30 DAY). | Date filters can reduce the amount of data scanned. |
| Exclude the newest days from final reporting | For routine daily reporting, consider ending the window three days before the current date. | The two tables export separately and reporting data can change or arrive late. |
| Avoid deprecated fields | Check Google’s current documentation before reporting on older search appearance fields. | Fields such as is_tpf_faq no longer provide useful recent data. |
| Annotate known data anomalies | Join your reports to a table of documented anomaly dates. | Logging errors can distort historical trends. |
Google’s sample queries are useful starting points, but check them before adding them to your reporting system.
Four things deserve particular attention.
- Sort top-performing queries correctly. A query described as a list of the most popular items should normally sort clicks or impressions in descending order using
DESC. - Use Pacific Time. Replace
CURRENT_DATE()withCURRENT_DATE('America/Los_Angeles')where the report must align with Search Console dates. - Do not rely on obsolete FAQ fields. The
is_tpf_faqfield no longer supports useful recent FAQ rich result analysis. - Handle brand and anonymised queries separately. Brand classification requires a pattern that matches your brand names and common variations. Anonymised queries should not accidentally be counted as non-brand traffic.
Search Console’s branded-query filter is not included in the BigQuery export. You must maintain your own classification rules, so the results may not match the interface exactly.
Step 6: Monitor the Export Like a Production System
One of the most important deadlines is about a week.
If you fix an export problem within that period, Search Console can still include the missed data in a later export. Leave it too long and the missing dates may be lost permanently. After about a month of failed exports, the entire export stops.
Use a monitoring schedule with clear responsibilities.
| Frequency | What to check | Owner |
|---|---|---|
| Daily, automatically | Run the export alert query and send an email if recent dates are missing, incomplete or too old. | Developer |
| Whenever an export error email arrives | Investigate the error, correct access or billing issues, use Search Console’s test function where applicable, and check the next export. | Developer and property owner |
| Weekly | Run the export health query and identify any dates that Google has revised. Rebuild affected summary tables. | Analyst |
| Monthly | Compare BigQuery costs with your estimates. Investigate unexpected charges and review Cloud Logs Explorer if necessary. | Data owner |
| Quarterly | Review permissions, billing, property ownership and the accounts receiving export notifications. | Cloud administrator and SEO lead |
| Whenever someone proposes changing the export schema | Keep the export tables unchanged and create custom reporting tables in seo_reporting. | Developer |
The health check is particularly important because Search Console only displays the latest export attempt in its settings. Failed attempts are not recorded in ExportLog.
That table is therefore an important record of the successful exports you can use to check data completeness and identify revisions.
Step 7: Stop, Restart or Move the Export Carefully
There may be occasions when the export needs to be stopped, restarted or moved to another location.
Stopping the Export
In Search Console’s bulk data export settings, select Deactivate export.
Exports normally stop within 24 hours, meaning another export may arrive before they stop completely.
If exports must stop immediately, Google documents an alternative involving the removal of the Search Console principal’s permissions from your Cloud project.
Existing tables are not deleted automatically.
Restarting the Export
Follow Google’s setup process again and use the same project ID and dataset location if you want to continue with the existing tables.
If the tables still exist, new data can be added to them. Check that the required permissions and other settings have not changed.
Moving to Another Dataset Location
BigQuery does not support moving the bulk export through a simple location setting.
Google documents a workaround:
- Stop the export and wait 24 hours.
- Confirm that the last export completed successfully.
- Copy the dataset to the new location.
- Start a new export using the new location.
You may miss a day or two of data during the move. Use the API to fill gaps where possible and label those records as API data.
Step 8: Roll Out the Data to Each Team
Once the pipeline works reliably, decide what each team needs to do with the data.
| Team | What the data provides | Recommended first actions |
|---|---|---|
| Content | Page performance, query coverage and queries that appear across several URLs | Review overlapping queries monthly and decide whether pages should be consolidated, differentiated or left unchanged. Use complete query sets for topic modelling and content planning. |
| Marketing and reporting | Daily search totals, brand and non-brand analysis, anonymised query reporting and annotated trends | Rebuild reporting using the site table and summary tables. Connect search demand with analytics and CRM data where the relevant identifiers and tracking permit it. |
| Development and data | The export pipeline, access controls, cost limits, reporting tables and monitoring | Own setup, cost controls and alerts. Document the pipeline in the team’s technical runbook. |
| Leadership and data owner | Retention, budget, access and business reporting | Approve the initial setup decisions and review costs, access and retention policies quarterly. |
The real benefit comes when the data is used consistently across the business, rather than remaining in BigQuery as a technical project.
5. Developer Toolkit
The following examples show how to plan costs, monitor the export and create summary tables for regular reporting.
Cost and Settings Planner
The bulk_export_planner.py utility is intended to estimate storage and query costs from your daily row counts. It also helps you decide on retention and query limits.
The calculation uses the published BigQuery pricing assumptions and estimates row sizes from the export schema and average text lengths.
Once the first export has arrived, count the rows in each table for a representative date and use those figures as the planner’s inputs.
For example:
python3 bulk_export_planner.py \
--site-rows 2000000 \
--url-rows 5000000 \
--refreshes-per-day 24 \
--days-scanned 180 \
--project acme-seoThe example output is:
Bytes per row site 110 url 203
New data per month 34.51 GiB
Stored at 25 months 862.6 GiB -> storage about $14.30 a month
Dashboard on raw URL table: 102.09 GiB per refresh,
71.78 TiB a month -> about $442.39 a month
Same dashboard on a daily summary table:
7.190 TiB a month -> about $38.69 a monthThese figures are estimates, not guaranteed billing amounts. Confirm the actual bytes processed using BigQuery’s dry-run feature and check your billing reports.
For the example project, the planner recommends the following settings.
Partition expiration: 761 days
Per-query cap for analysts: --maximum_bytes_billed=220117073920The example retention commands are:
ALTER TABLE `acme-seo.searchconsole.searchdata_site_impression`
SET OPTIONS (partition_expiration_days = 761);
ALTER TABLE `acme-seo.searchconsole.searchdata_url_impression`
SET OPTIONS (partition_expiration_days = 761);Use your actual project and dataset names. Check the consequences of partition expiration before applying the setting, because older partitions may become eligible for deletion.
The planner also needs to account for the minimum retention requirements of your own reporting and the free allowances included in BigQuery’s pricing.
SQL Pack: Monitoring and Reporting Queries
The following SQL examples cover five useful tasks: checking export health, alerting on failures, reporting daily totals, separating brand from non-brand queries and building a page-level summary table.
Replace your-project, searchconsole and seo_reporting with the names used in your own Google Cloud setup.
Run each query as a dry run first. Then compare the results for one day with Search Console’s Performance report before scheduling anything.
1. Export Health: Find Missing, Incomplete and Revised Days
This query checks the export log for recent dates and flags potential problems.
WITH days AS (
SELECT
data_date,
COUNT(DISTINCT namespace) AS tables_exported,
MAX(epoch_version) AS latest_epoch
FROM `your-project.searchconsole.ExportLog`
WHERE data_date >= DATE_SUB(
CURRENT_DATE('America/Los_Angeles'),
INTERVAL 45 DAY
)
GROUP BY data_date
),
flagged AS (
SELECT
*,
DATE_DIFF(
data_date,
LAG(data_date) OVER (ORDER BY data_date),
DAY
) - 1 AS missing_days_before
FROM days
)
SELECT
data_date,
tables_exported,
latest_epoch,
missing_days_before,
CASE
WHEN tables_exported < 2 THEN 'HALF EXPORTED'
WHEN missing_days_before > 0 THEN 'DAYS MISSING BEFORE THIS DATE'
WHEN latest_epoch > 0 THEN
'REVISED BY GOOGLE: REBUILD SUMMARIES FOR THIS DATE'
ELSE 'OK'
END AS status
FROM flagged
ORDER BY data_date DESC;Each date should normally have successful exports for both performance tables.
A revised epoch_version indicates that Google updated previously exported data. If your reporting tables contain the old values, rebuild the affected summaries.
2. Export Alert: Run Daily With Email Notifications
This query is designed for a scheduled query with email notifications enabled.
It checks whether the latest export is delayed and whether the expected recent dates have both tables.
WITH days AS (
SELECT
data_date,
COUNT(DISTINCT namespace) AS tables_exported
FROM `your-project.searchconsole.ExportLog`
WHERE data_date >= DATE_SUB(
CURRENT_DATE('America/Los_Angeles'),
INTERVAL 14 DAY
)
GROUP BY data_date
)
SELECT IF(
MAX(data_date) IS NULL
OR MAX(data_date) < DATE_SUB(
CURRENT_DATE('America/Los_Angeles'),
INTERVAL 3 DAY
)
OR COUNTIF(
data_date <= DATE_SUB(
CURRENT_DATE('America/Los_Angeles'),
INTERVAL 3 DAY
)
AND tables_exported >= 2
) < 12,
ERROR(
'Search Console bulk export: late, missing or half-exported days. Run the health query.'
),
'ok'
) AS export_check
FROM days;The example expects both tables to be present for each of the 12 dates from 14 days ago through three days ago.
The date window leaves recent days out of the check to account for normal export delays. Adjust the window if your export schedule or reporting requirements call for a different approach.
3. Daily Web Totals for Comparison With Search Console
This query calculates daily clicks, impressions, CTR and average position using the site-level table.
SELECT
data_date,
SUM(clicks) AS clicks,
SUM(impressions) AS impressions,
SAFE_DIVIDE(
SUM(clicks),
SUM(impressions)
) AS ctr,
SAFE_DIVIDE(
SUM(sum_top_position),
SUM(impressions)
) + 1 AS avg_position
FROM `your-project.searchconsole.searchdata_site_impression`
WHERE data_date BETWEEN
DATE_SUB(
CURRENT_DATE('America/Los_Angeles'),
INTERVAL 30 DAY
)
AND DATE_SUB(
CURRENT_DATE('America/Los_Angeles'),
INTERVAL 3 DAY
)
AND UPPER(search_type) = 'WEB'
GROUP BY data_date
ORDER BY data_date;The query aggregates all the metrics and uses the correct average-position formula. It also applies a date filter to reduce the amount of data scanned.
4. Separate Brand, Non-Brand and Anonymised Queries
This example groups query performance into three categories.
SELECT
CASE
WHEN is_anonymized_query IS TRUE
OR IFNULL(query, '') = ''
THEN 'anonymised'
WHEN REGEXP_CONTAINS(
query,
r'(?i)acme|acmee|acme widgets'
)
THEN 'brand'
ELSE 'non-brand'
END AS query_class,
SUM(clicks) AS clicks,
SUM(impressions) AS impressions
FROM `your-project.searchconsole.searchdata_site_impression`
WHERE data_date BETWEEN
DATE_SUB(
CURRENT_DATE('America/Los_Angeles'),
INTERVAL 30 DAY
)
AND DATE_SUB(
CURRENT_DATE('America/Los_Angeles'),
INTERVAL 3 DAY
)
AND UPPER(search_type) = 'WEB'
GROUP BY query_class
ORDER BY clicks DESC;Replace the example brand pattern with your business name, common misspellings and relevant product names.
Review the pattern periodically. A simple regular expression may not capture every relevant brand query, particularly when product names have several meanings.
5. Create a Daily Page Summary Table
For routine reporting, build a smaller table that contains only the dimensions and metrics your dashboards need.
CREATE TABLE IF NOT EXISTS
`your-project.seo_reporting.page_daily` (
data_date DATE,
url STRING,
search_type STRING,
country STRING,
device STRING,
clicks INT64,
impressions INT64,
sum_position FLOAT64
)
PARTITION BY data_date
CLUSTER BY url
OPTIONS (require_partition_filter = TRUE);The following scheduled query rebuilds the last ten days of data.
BEGIN TRANSACTION;
DELETE FROM `your-project.seo_reporting.page_daily`
WHERE data_date >= DATE_SUB(
CURRENT_DATE('America/Los_Angeles'),
INTERVAL 10 DAY
);
INSERT INTO `your-project.seo_reporting.page_daily`
(
data_date,
url,
search_type,
country,
device,
clicks,
impressions,
sum_position
)
SELECT
data_date,
url,
UPPER(search_type),
country,
device,
SUM(clicks),
SUM(impressions),
SUM(sum_position)
FROM `your-project.searchconsole.searchdata_url_impression`
WHERE data_date >= DATE_SUB(
CURRENT_DATE('America/Los_Angeles'),
INTERVAL 10 DAY
)
GROUP BY
data_date,
url,
UPPER(search_type),
country,
device;
COMMIT TRANSACTION;Rebuilding the recent ten-day window helps account for late exports and updates to recent data. Running the scheduled query again will not double-count the same rows because it deletes the relevant dates before inserting the refreshed results.
If the health query identifies a revised date outside that window, rebuild the affected date separately.
The summary table stores sum_position rather than an average. This allows dashboards to calculate average position correctly across any chosen date range.
The require_partition_filter setting also makes BigQuery reject queries against this table that omit a date filter. This helps reduce the risk of accidentally scanning the entire table.
Keep these controls on your own reporting tables. Do not add custom columns to Google’s export tables.
6. What We’re Watching Next
Bulk data export is already useful for detailed SEO reporting. Several developments could make it more powerful, but teams need to keep their data pipelines flexible.
Generative AI Data in Search Console Exports
Google’s data anomaly documentation now includes events affecting generative AI in Search and Discover.
However, the documented bulk export schema lists six search types: web, image, video, news, Discover and Google News. It does not currently provide a separate search type specifically for AI Overview or AI Mode performance.
We are watching for additional search types or search appearance fields that make AI search performance easier to identify in BigQuery.
When Google adds new fields, reporting pipelines should be updated deliberately. Teams should not wait until a dashboard breaks or a report starts showing unexpected results.
Changes to Google’s Export Schema
Google can add search appearance fields or retire fields that no longer provide useful data.
For example, the is_tpf_faq field stopped being useful for recent reporting after FAQ rich results were removed from Google Search on 7 May 2026.
Queries that select only the fields they need are generally easier to manage than those built around SELECT *. The latter can scan additional columns or require changes when the schema evolves.
Review Google’s table documentation regularly and test your reporting queries after schema changes.
Revisions to Previously Exported Data
Google sometimes corrects data after it has already been exported.
The August 2026 generative AI reporting issue is an example. Google confirmed that the missing data had been restored in the Performance report.
Where a correction is reflected in the bulk export, the epoch_version field in ExportLog can help identify affected dates.
That is why a health check needs to look for revisions, not just missing rows.
When Google updates a date, rebuild the reporting tables that contain the previous figures so they remain consistent with the latest export.
A Branded Query Field
Search Console offers branded and non-branded query filtering in its interface, but the branded-query classification is not included as a dedicated field in the bulk export.
As a result, brand reporting in BigQuery still depends on the patterns you maintain yourself.
Those patterns may not perfectly match Google’s classification. A dedicated brand field in the export would make this type of analysis easier to manage at scale.
BigQuery Pricing for AI Functions
Standard SQL queries have relatively predictable costs when you control the amount of data they scan.
AI-powered forecasting and classification introduce additional costs that need to be assessed separately from storage and ordinary queries.
Google’s pricing documentation should be checked before using BigQuery ML or other AI functions for large-scale query classification or forecasting.
As the available models and pricing arrangements change, update your cost estimates rather than assuming that the cost of AI processing is included in your normal BigQuery budget.
7. About Szymaniak Digital
Szymaniak Digital is a UK enterprise SEO and AI search consultancy helping businesses improve search visibility, understand customer demand and grow through organic search.
We set up and audit Search Console bulk exports, build reporting systems in BigQuery and use detailed query data for analysis that standard Search Console reports cannot support at the same scale.
This includes large-scale keyword clustering, query-to-URL mapping, content analysis and research into how people discover businesses through Google Search and AI search platforms.
If your bulk export is not running, or you are unsure whether it has been working correctly since setup, we can review your ExportLog, query costs and reporting queries.
The aim is to establish which data you have, where gaps exist and what you can reliably learn from it.
