JAN 23, 2023
Our Story
In my last startup, every day we were receiving CSV files from various digital out-of-home screen owners in a specific template defined by us. Even though the template was well-defined, we used to get a lot of errors in the CSV files shared, and getting it cleaned with the data provider was a nightmare.
Each day we received approximately 5,00,000 screen data updates, with Price Changes, Dimension Data. Cleaning, and formatting the Data as per my DB and then uploading it into the DB was a pain and consumed a lot of my engineering hours. and I was looking to automate the complete importing workflow.
Location data importing
I checked the Open-Source community for any full-stack solutions but most of the solutions were available in silos and provided no end-to-end functionality which could seamlessly integrate with my product. Most of the CSV importing is limited to browser-level data import.
After deep diving more into the enterprise solutions, I discovered the current solutions available in the market pose the serious risk of Data sharing which may lead to privacy lapses, and vendor lock-in.
So focused on this problem area, envisioned building an open-source product “Yo-Bulk” which is a scalable, open-sourced tool, that can be embedded with your product or SaaS and can automate complete data importing workflow.
While in my second company LocTruth, I faced a new issue, I was importing Location Data and for Geospatial data I was unable to find a tool that validates Geospatial data not even able to parse and understand geo-specific formats like KLM, geo-json.It's a massive problem to validate a CSV file which has geometry like attributes and import it to postGIS.
Hence, I got convinced there has to be a Generic CSV Importer for SaaS applications which would be built by an Open-Source community to cater to the needs of data received from different Domain like adtech, martech, finance, retail,Geo-spatial etc.
On 1st week of Nov 2022, we conceptualized this Idea, and started development. We chose tech Stack as Next.js, and Mongo DB as our DB and within 12 weeks we are ready..
YoBulk as the name suggests wants to fasten the CSV Importing process and this can be a path breaker for a Developer and for a non-technical user as well.