| .. | ||
| docker-compose.yml | ||
| README.md | ||
| source_dune_analytics.yaml | ||
Airbyte
Usage
Airbyte allows for quickly tying in per-canned and custom data sources to the data lake. In this case, airbyte was able to quickly trigger the execute command on Dune, grab the execution id, then pull the data when polling shows that it was ready. It then pushes the data as raw into the database, keeping track of any errors and avoiding duplicates.
Demo
the demo is at Airbye
username: airbyte
password: password
(i kept the defaults)
- note the airbyte instance and the open-metadata instance must be seperate machines/vms/lxcs with the supplied docker-compose files)