Data systems designed for the questions you'll ask in year three
Database and data systems work is the foundation layer: designing schemas that model your business correctly, keeping queries fast as data grows, and building the pipelines that turn operational records into reports people trust. It removes two slow-motion failures — the database that gets a little slower every month, and the meeting where two reports disagree and nobody knows which is right.
Willowark designs from the domain outward. We model the entities and events your business actually has, choose PostgreSQL by default because it earns that position, and design indexes around the queries you will actually run — including year-three questions like 'show me margin by customer by quarter' that a purely transactional schema answers only with pain.
Software EngineeringHow the work gets done
The same way every time: scope, build, hand over.
Transactional and analytical workloads want different shapes, so we separate them: normalized schemas with real constraints for the system of record, and read models or a warehouse fed by change data capture or scheduled ELT for reporting — so a heavy Monday-morning dashboard never locks up order entry. Migrations are versioned and reversible, large tables get partitioning strategies before they need them.
Operations are part of the deliverable. Backups only exist if restores are rehearsed, so we rehearse them. Slow-query monitoring catches regressions while they are cheap. Data-quality checks — row counts, referential integrity, freshness — run on a schedule and alert when reality drifts, keeping reports trusted long after launch.
Scoping data work means collecting the questions before touching the schema. We ask operations, finance, and leadership what they need to know, then trace each question back to the records that answer it. That usually surfaces the gaps — the timestamp nobody stores, the status that gets overwritten instead of logged — that make year-three reporting painful. The trade-offs we weigh are normalization versus query convenience, strict constraints versus a faster first release, and a shared database versus a separate reporting store. We lean toward the stricter choice for the system of record; it is cheaper to relax a constraint later than to clean up data that never had one.
Data systems go wrong slowly, which is what makes them dangerous: a nightly job that starts finishing at 7 a.m. instead of 3, a table that quietly doubled, a report someone modified by hand and nobody noticed. We design against that with monitoring on job duration and table growth, pipelines that are idempotent so a rerun repairs rather than duplicates, and metric definitions kept in version control where a change is visible. Handover includes the schema documentation, the migration history, restore procedures your team has actually executed, and a data dictionary written for analysts rather than engineers. Success is a database nobody thinks about and reports nobody argues with.
Scope it in writing
What we agree before work starts
- Schema design with documented modeling decisions and constraints
- Query and index optimization against your real workload
Build with checkpoints
Working results, not slide decks
- Reporting layer or warehouse with pipelines from operational systems
- Versioned migration framework with rollback paths
Hand over something you own
Documentation, source, and training
- Backup and restore procedures, rehearsed and documented
- Data-quality monitoring with freshness and integrity checks
Sound familiar?
Where database & data systems earns its keep.
A five-year-old application database where every screen has slowly gotten slower
Finance and operations pulling numbers from different systems that never quite match
A reporting query that takes eleven minutes and gets pasted into a spreadsheet weekly
A growing product about to outgrow the database its no-code platform started on
Ask about Database & Data Systems
Describe the problem. Get a straight answer.
One line is enough. An engineer replies within a business day.
Related work
AI operating software for training operations
Components:
- Schedulers (training ops): The people running the training operation.
- Operating software (AI assistance): The operating software, with AI assistance built in.
- Cloud services (distributed): Distributed cloud services behind the application.
- Training game (learning): The training game the software connects to.
Connections:
- Schedulers to Operating software
- Operating software to Cloud services
- Operating software to Training game
A global automotive manufacturer's training operation · Automotive
AI operating software for training operations
AI operating software for the manufacturer's training schedulers — removing manual scheduling labor and logistics tracking, saving hundreds of hours per year — plus a training video game built to help trainers perform better. Used across the manufacturer's training organization.
Read the case study →Common questions
Asked before every database & data systems project.
SQL or NoSQL?
SQL, until a specific access pattern says otherwise. Relational databases with real constraints protect data integrity in ways application code never fully replicates, and PostgreSQL now covers document, full-text, and even vector workloads credibly. NoSQL earns its place for genuine cases — extreme write throughput, flexible-by-design documents — as an addition, rarely as the foundation.
Can you fix performance without a migration?
Usually, yes. Most database pain traces to missing indexes, queries that fetch far more than they use, and contention from long transactions — all fixable in place. We measure first with query-level statistics, fix the top offenders, and verify against production-shaped load. Re-platforming is the answer occasionally, not routinely.
How do we get one version of the truth for reporting?
By making the pipeline explicit. Operational systems feed a reporting store on a defined schedule, metrics are defined once in that layer rather than in each person's spreadsheet, and freshness is visible so nobody unknowingly presents stale numbers. Report disagreements almost always trace to informal extracts — the fix is architectural.
Can you work with the database we already have, even if it isn't PostgreSQL?
Yes. SQL Server, MySQL, Oracle, and the databases embedded in ERP and line-of-business products are all common starting points, and the principles — measure the workload, fix the worst queries, add the missing indexes, separate reporting from transactions — apply regardless of vendor. We recommend a platform change only when the current one genuinely blocks what you need, and never as the opening move. Most improvement is available in place.
Do we need a data warehouse?
Not necessarily. A read replica or a set of materialized views on your existing database covers reporting for many businesses well before a warehouse is justified. A warehouse earns its place when reporting queries interfere with operations, when data must be combined from several systems, or when history needs to be kept in a shape the operational schema does not preserve. We start with the lightest structure that answers the questions and move up when the data says to.
Where this sits
Database & Data Systems, inside a software engineering system.
The lit component is the part of the system this service delivers; the rest is what it has to work with.
Hover or focus a component to see what it is and what it talks to. Arrow keys move between them.
Users reach a web app that talks to an API over a database, with background workers for the slow work and integrations to the systems already in use.
Components:
- Users: Staff, customers, or both.
- Web app: The interface: fast, accessible, works on a phone.
- API: Typed, versioned, authenticated.
- Database: Postgres, usually. Backed up, migrated, monitored.
- Workers (jobs, queues): Reports, syncs, emails — anything that should not block a click.
- Integrations (existing tools): Accounting, CRM, email, payments.
Connections:
- Users to Web app over REST
- Web app to API over REST
- API to Database over SQL
- API to Workers over events
- Workers to Integrations over webhook, both directions
Strategy. Software. Systems.
Have a system that should exist?
Tell us what your operation is doing manually, what isn't connected, or what you're trying to build. We'll tell you plainly whether and how we can help.