The new pipeline, Yandex.Direct → Google BigQuery, is out now as a public beta! Now you can upload full raw cost data on your Yandex.Direct ads straight to Google BigQuery, no workarounds or limitations.
The new pipeline also lets you:
- Upload any volume of data each day into any Google BigQuery project you have account access to
- Collect data from many advertising accounts into a single partitioned table to quickly build reports on the united account data
- Get historical cost data for any time period (via a request to OWOX BI support)
Create the pipeline now and try it out for free and with no limits throughout the open beta period.
Note: The old Google BigQuery export feature in the Yandex.Direct → Google Analytics pipeline will become unavailable on August 31, 2019. After that, OWOX BI will be uploading cost data from Yandex.Direct to Google BigQuery only via the new pipeline.
How to set up the pipeline
Note: This pipeline is available for the subscription plans Basic, Business, and Enterprise.
1. On the OWOX BI main page, click Create pipeline:
2. As a source, select Yandex.Direct:
3. As a destination, select Google BigQuery:
4. Select or add new access to a Yandex.Direct account where you wan to export ad cost data from:
5. Select or add new access to a Google BigQuery account where you want to store the exported data:
6. Select a Google BigQuery project and create a dataset you want to upload your data to (or create an existing one):
Note: To setup the data collection successfully, your Google account must have the “BigQuery Admin” role in the project where you want to collect data to. Without this permission, BigQuery won’t let you upload the data.
7. Specify the date that marks the beginning of the period you want to upload cost data for:
You can schedule the start of the import on a date in the future or select a past date to upload historical data from the Yandex.Direct account.
If you need to upload historical data for the period that is larger than it’s available in the interface, set the furthest possible past date and create the pipeline. Then, write to our support at email@example.com, specify the period you need the data for, and we’ll upload the rest of the historical data you need.
Also on this step, specify the VAT rate to exclude from the exported costs. You need this to have your Yandex.Direct cost data consistent with the cost data form other services where VAT is being excluded automatically.
8. Click Create pipeline.
Done! The pipeline will be uploading data to a partitioned table in the selected BigQuery dataset. The data will be uploaded daily — for the previous day.
OWOX BI will also update the data already uploaded to BigQuery if the data in the Yandex.Direct account would change retrospectively. The update period for historical data is 21 days. This means that 21 days after the import, the data in BigQuery will be totally up-to-date.
The table data schema is in this article.
How exactly the Yandex.Direct → Google BigQuery pipeline is better than the BigQuery export feature for the Yandex.Direct → Google Analytics pipeline
On August 31, 2019, the old BigQuery export feature will become unavailable in OWOX BI. The Yandex.Direct → Google BigQuery pipeline is conceived as the more effective replacement for the old BigQuery export feature. Our goal with this pipeline is to provide you with a more stable and reliable tool for uploading cost data to Google BigQuery.
Here are the major benefits of the new pipeline over the old export functionality.
The old export method lets you upload cost data only to the datasets in the Google BIgQuery projects created on the OWOX BI side.
The new Yandex.Direct → Google BigQuery pipeline lets you upload cost data to any BigQuery project you will provide account access to.
Simple work with data from many Yandex.Direct accounts
With the old method, if you had several Yandex.Direct account, you had to collect data from each of them to separate tables. To build reports on all these data, you had to unite it on your side.
The new pipeline collects data from all your Yandex.Direct into a single partitioned table. You no longer need to unite the data from several accounts manually.
Even while in public beta, the Yandex.Direct → Google BigQuery pipeline provides a much more stable data upload than the old BigQuery export feature for the Yandex.Direct → Google Analytics pipeline. This is especially crucial if you operate large volumes of data: the pipeline can handle gigabytes of data each day.
What happens to data already exported to Google BigQuery from the Yandex.Direct → Google Analytics pipeline
The data uploaded via the BigQuery export feature for the Yandex.Direct → Google Analytics pipeline will remain in the Google BigQuery datasets on the OWOX BI side until September 30, 2019. On that date, these datasets will be deleted.
These data won’t be transferred to the Google BigQuery project you set the new Yandex.Direct → Google BigQuery pipeline’s data collection to.
However, if you’re subscribed to the plans Basic, Business, and Enterprise, you can upload any historical data you need from your Yandex.Direct account. To do this, write to us at firstname.lastname@example.org and specify the period for which you want to retain the historical data.
What happens to the Yandex.Direct → Google Analytics pipeline
Ad cost data import to Google Analytics continues to work as intended. The Yandex.Direct → Google BigQuery and Yandex.Direct → Google Analytics pipelines are different tools that work independently.
Note that Yandex.Direct → Google BigQuery is now in open beta and the data upload may be unstable at times. Yet, the OWOX BI team is actively optimizing the pipeline and your feedback would be highly helpful in speeding up this process.
If you have encountered any issues in setting up the pipeline or with data collection, please let us know about these difficulties by writing at email@example.com. We’ll be happy to help you with setting up the pipeline and any feedback would help us make the new pipeline even better.