"We're thrilled to have you onboard!"
For any SaaS business, a new customer is a cause for celebration. But for the engineering team, it often kicks off a familiar, time-consuming ritual: data onboarding. Your new customer has a treasure trove of data, but it's in their format. Now, it's your team's job to wrangle it, reshape it, and load it into your system.
This process is a silent killer of productivity. It involves endless CSV columns, strangely nested JSON, back-and-forth emails to clarify field meanings, and a growing collection of one-off, brittle scripts. While your team is bogged down in this ETL mire, they aren't building your core product.
There has to be a better way. It's time to stop treating data onboarding as a manual chore and start treating it like any other automated, scalable part of your infrastructure: as code.
If you've ever been tasked with integrating a new customer's data, this probably sounds familiar:
This cycle is not scalable. It frustrates your customers with long onboarding times and burns out your engineers with repetitive, low-impact work.
Imagine defining these complex data wrangling tasks not with brittle scripts, but with a simple, declarative set of rules. This is the core idea behind Data Transformation as Code.
Instead of writing how to transform the data step-by-step, you simply declare what the end result should look like. Your transformation logic becomes a simple configuration file that can be version-controlled, reused, and tested just like the rest of your application code.
This is precisely what we built at transform.do. We turn complex ETL jobs into simple, powerful API services. You define the rules, and our AI-powered agents handle the execution.
Let's see how you could automate the customer onboarding scenario we described earlier. Instead of a custom Python script, you can use the transform.do API to define and execute the entire workflow.
Here’s how you could reshape a customer's user data to fit your system's schema in just a few lines of TypeScript:
import { Agent } from "@do/sdk";
// Initialize the transformation agent
const transform = new Agent("transform.do");
// Your customer's source data
const sourceData = [
{ "user_id": 101, "first_name": "Jane", "last_name": "Doe", "join_date": "2023-01-15T10:00:00Z" },
{ "user_id": 102, "first_name": "John", "last_name": "Smith", "join_date": "2023-02-20T12:30:00Z" }
];
// Define your transformation rules as a simple object
const transformations = {
targetFormat: "json",
rules: [
{ rename: { "user_id": "id", "first_name": "firstName", "last_name": "lastName" } },
{ convert: { "join_date": "date('YYYY-MM-DD')" } },
{ addField: { "fullName": "{{firstName}} {{lastName}}" } }
]
};
// Execute the transformation with a single API call
const result = await transform.run({
source: sourceData,
transform: transformations
});
// The data is now perfectly formatted for your system
console.log(result.data);
/*
[
{ "id": 101, "firstName": "Jane", "lastName": "Doe", "join_date": "2023-01-15", "fullName": "Jane Doe" },
{ "id": 102, "firstName": "John", "lastName": "Smith", "join_date": "2023-02-20", "fullName": "John Smith" }
]
*/
In this example, we declaratively:
This transformation definition is simple, readable, and reusable. For your next customer, you just tweak the rules, not write a whole new script.
This approach extends far beyond the initial onboarding. Because each transformation is an API endpoint, you can build powerful, automated data pipelines.
Your development team's time is your most valuable resource. Don't let it be consumed by manual, repetitive data integration tasks. By automating customer data onboarding, you can:
Stop the cycle of manual data wrangling. Start building scalable, robust, and automated data workflows.
Ready to transform your data onboarding process? Get started with transform.do today.