Data Pipeline Vendor Lock-In Is Built One Connector at a Time

Most conversations about vendor lock-in are about contracts. Term length, exit clauses, egress fees, proprietary formats. Those are the visible part, and they are the part a procurement team can negotiate.

The part nobody negotiates is the one that decides whether leaving is possible at all. If you gave notice on Monday, what would you have to rebuild before you could run again?

That number is your lock-in. Everything else is a line item.

1. Lock-in is a rewrite estimate, not a contract term

The reason the contract framing misleads is that the expensive thing is never the data. Data moves. Storage is a solved problem, formats are increasingly open, and a bulk export is a weekend at worst.

What does not move is the logic that turned a source into something usable. The connector that knows this API paginates by page number. The rule that a null customer reference means the record predates the migration and should be routed, not dropped. The knowledge that this vendor sends dates in two formats depending on which of their systems generated the row.

None of that is data. All of it is code, or worse, configuration inside somebody's product. And the cost of leaving is the cost of writing it again, by people who no longer remember why it was written the first time.

So the useful question is not what the exit costs in euros. It is how many engineering days sit between the decision to leave and the first correct run afterwards.

2. Why the ingestion layer accumulates it faster than anything else

Three properties combine here that do not combine anywhere else in the stack:

• You add connectors one at a time, and each one is small: Nobody convenes a review to add the 14th source. The decision is made by whoever had the ticket, in an afternoon, using whatever the platform makes easy. Fifty afternoons later the accumulation is significant and no single afternoon was wrong.
• Every connector encodes decisions that nobody records: What happens to a row with no primary key. Whether a field that arrives as a string where it was an object gets coerced, rejected, or split. How nested arrays are flattened. What counts as a failed record. Each of these is a semantic choice with downstream consequences, and when a managed platform makes it for you, it is made silently and correctly according to that platform's opinion.
• The decisions become load-bearing without anyone deciding they should: The moment a dashboard, a model, or a finance report depends on a number, the policy that produced that number is frozen. Changing it later is not a refactor, it is a discrepancy someone has to explain.

Put together: the ingestion layer is where your business logic goes to hide. Which is another way of saying the ingestion layer is where the decisions with consequences live: if you cannot read them, you do not own them.

3. The inventory, and the answer you do not want

Run those five questions against your own stack, honestly, and the total tells you what you would rewrite.

Every row where your honest answer is in the right-hand column is work you would do again. Count it in engineering days. That total, not the contract, is your switching cost.

Two things usually surprise people running this for the first time. The connectors are rarely the biggest number, because a connector is a bounded piece of work. The quality rules are, because they accumulated over years, they were never written down anywhere else, and the person who knew why rule 31 exists has left.

4. Three things that look like portability and are not

Each of these appears in vendor material and each is true in a narrow sense that does not survive contact with an actual migration.

• "We use open formats:" Your storage is portable. Parquet is Parquet and Iceberg is Iceberg. The logic that wrote into it is not, and the logic is what you would rebuild. Open storage formats solve the problem that was already easiest.
• "You can export your configuration:" A YAML file you cannot execute anywhere is a description of your pipeline, not your pipeline. Ask what runs it after the export. If the answer is only the product you are leaving, you have exported a document.
• "It is open source:" Self-hostable and portable are different properties. An open-source connector still targets that project's interfaces, its state model, its idea of what a sync is. You have removed the licence risk and kept the rewrite. That is worth something, and it is worth less than it sounds.

The test that cuts through all three: name the runtime you would move to, and ask what percentage of the artifact runs there unmodified.

5. What changes in January 2027, and what it does not fix

The EU Data Act makes portability a binding right rather than a commercial courtesy. Its switching provisions have applied since September 2025, and from 12 January 2027 providers are generally no longer permitted to charge for switching, including data egress and transit charges.

This is a real change and it is worth planning around. It is also narrower than the headlines suggest, for a reason that follows directly from section 1.

The law removes the fee. It does not remove the rewrite.

After January 2027 your exit gets cheaper in euros and stays exactly as expensive in engineering days. A right to leave is worth what your technical ability to exercise it is worth, and that ability is decided by choices you already made, one connector at a time, in afternoons nobody reviewed.

Which is the argument for running the inventory in section 3 now rather than in 2027. The regulation gives you the door. Whether you can walk through it is a property of your own repository.

6. Measuring your own number in an afternoon

You do not need a full audit, you need an estimate good enough to argue with.

• Pick your 3 most business-critical sources: Not the 3 most complex, the 3 whose numbers appear in front of someone who matters.
• For each, list what you would have to rebuild to produce the same output on a different runtime. Connector, transformations, quality rules, schedule, alerting, and the historical backfill.
• Estimate it in engineering days, using your own delivery history rather than a benchmark.
• Multiply by your source count, with a discount for the ones that are genuinely simple. The result is wrong in the third digit and right in the order of magnitude, which is all a decision needs.
• Write the number down and date it. Then look at it again at the next renewal, which is the only moment anyone has leverage.

The teams that do this find one of two things. Either the number is small, in which case a managed platform is a good deal and they should stop worrying about it. Or the number is large and nobody knew, which is the situation this article is about.

What to take away

Lock-in in a data platform is not a clause you agreed to. It is an accumulation you never measured, built one connector at a time by people making reasonable local decisions.

The measurement is a rewrite estimate, in engineering days, for the sources that matter. The five inventory questions get you there in an afternoon: where do the connectors live, where do the transformations live, where do the quality rules live, who owns the orchestration definition, and can the history be reproduced from a repository.

The regulatory change arriving in January 2027 removes the exit fee. It leaves the rewrite untouched, which makes the rewrite the only number worth knowing.

PrettyWhale.ai produces the ingestion deliverable as portable code the client owns and runs: the pipeline, its tests, its quality transformations, its output schema, its configuration and its documentation. There is no vendor in the data path, no volume meter, and nothing to export, because nothing was ever held elsewhere. The trade is explicit: you own the operation, and in exchange the decisions that shape your data are in a repository you can read.