From Excel Sheets to a Scalable Platform: A Pet Project That Grew
When people think of building business systems, they often imagine large teams, big budgets, and cloud-native stacks. But real engineering is often scrappy, incremental, and shaped by constraints.
This story isn’t about my day job. At the time, I was working full-time as a Solution Architect at a software company. That role gave me the space — and the necessity — to dive deep into modern infrastructure, orchestration tools, and integration patterns. On the side, I took on a small challenge for a friend’s SME. I treated it like a pet project, spending no more than 10 hours a week.
It’s also important to remember the timing: this was before today’s wave of LLMs and AI coding assistants. There was no “autocomplete that writes your microservice,” no AI workflow generator, no one-click orchestration setup. Every piece — from data pipelines to API gateway integrations — had to be built and stitched together manually.
Looking back today, it’s striking how different the experience would be. If I were building the same system now, I could lean on AI for scaffolding code, generating test cases, even suggesting integration patterns. It would save countless hours. But back then, the only leverage I had was pragmatic architecture choices, automation patterns, and persistence.
And yet, within 12 months, that side project evolved from messy Excel sheets into a working platform with microservices, workflow orchestration, and nearly 1,000 active collaborators.
Phase 0: First version of SMS Orders System
In the very beginning, there was no real system—just Excel files, phone calls, and SMS. Most collaborators didn’t use modern apps like Telegram or WhatsApp yet. Orders were placed by texting or calling, and every day the company had to assign someone to read incoming messages and manually type each order into Excel. It was repetitive, slow, and error-prone.
To solve this, I built a small desktop application. Using a simbox connected to the PC via COM port, the program could read SMS messages directly, parse them in a simple order format, and then automatically insert the orders into the appropriate Excel files. Now, collaborators only needed to send an SMS in the correct format, and the system would immediately create the order without human input.
This was my very first automation hack: turning SMS into a bridge for Excel, reducing manual work and laying the foundation for the next step—building a real application.
- Simbox + COM port: Physical device for SMS interception.
- Parser: Simple order format like “Order 123: 2x Sim 1GBperday”.
- Auto-creation: Orders pushed directly into the system without manual input.
Phase 1: Replacing Excel With a Real Application
The first step was simple but fundamental: Excel had to go. The client was a small SME with just four employees at that time, selling SIM cards and travel tours. Their main channel was telesales, with additional sales through agents and freelance collaborators. Everything ran on Excel—customer lists in multiple files, orders tracked in different sheets, and each person keeping their own copy. Duplicates, errors, and wasted time were inevitable.
I didn’t try to design a “perfect” system upfront. Instead, I used ASP.NET Maker to quickly generate a CRUD application. This included a centralized Customer module (add, edit, search), a unified Order module, and simple role-based access for telesales and admin.
For the first time, the company had:
- A single source of truth.
- Fast search and fewer duplicates.
- A real system they could trust.
This was the foundation. Without this step, nothing else would have been possible.
Phase 2: Cleaning Customer Data
With more collaborators and more data, lead quality became an issue. The telesales team often wasted time calling bad leads. To address this, I built a data pipeline that ingested and filtered customer data from multiple sources.
The pipeline used Spark + SQL Server to process raw data, applying criteria to score and tier customers. High-potential leads were then pushed into the telesales system, allowing the team to work smarter, not harder.
- Spark handled batch processing of customer records from various sources.
- SQL Server acted as a centralized data store for filtered and scored leads.
- Scoring criteria included factors like engagement level, contact history, and demographic relevance.
- Tiering allowed prioritization of leads based on their likelihood to convert.
Phase 3: Breaking the Monolith
The first ASP.NET Maker app was monolithic. As the business expanded — more agents, more collaborators, more integrations — it began to strain.
I transitioned to a microservices architecture:
-
Order Service
-
Partner Service
-
Commission Service
-
Pricing Service
To avoid drowning in CRUD work, I used gencode and a lightweight core CMS to automate boilerplate.
This allowed me to scale functionality quickly while keeping the codebase clean.
Phase 3.5: Splitting Applications for Different Roles
As collaborators and agents increased, managing everything in one big app became messy — especially access control. Designing a complex role-based permission system inside a monolith would have been overkill.
Instead, I chose a pragmatic approach: split the apps.
-
One app for Administration.
-
One app for Operations.
-
One app for Agents/Collaborators.
Each app handled only what its users needed. They were integrated together and authenticated through Keycloak, which became the single identity provider.
To make this work, I had to rewrite the authentication layer of my ASP.NET Maker–generated application so it could integrate seamlessly with Keycloak.
The benefits were huge:
-
Simpler apps, no complex internal permission logic.
-
A consistent login experience across all apps.
-
Security and session management centralized in Keycloak.
This decision saved me from endless headaches around role-based access, while keeping the architecture clean and scalable.
Phase 4: Integrating With External Partners
The biggest challenge came when the company needed to integrate with about 10 external service providers. Each partner had its own API quirks.
I solved this by introducing:
-
WSO2 API Manager for gateway, routing, and policy.
-
Elsa Workflow for orchestration.
I even had to customize Elsa to handle dynamic stores, because the legacy system had too many stores to rewrite.
This gave me:
-
End-to-end tracing of requests.
-
Built-in retry, timeout, and circuit breaker.
-
Ability to change routing rules without redeploying code.
It turned integrations from a nightmare into a manageable system.
Phase 5: Infrastructure on a Budget
The company couldn’t afford fancy infrastructure. I deployed on a minimal setup:
-
2 application servers (Docker).
-
1 database server.
-
1 backup server.
To guarantee data safety:
-
I implemented the Outbox Pattern + Kafka.
-
All transactions produced events.
-
Events were applied idempotently and replicated into the on-prem backup DB for reconciliation.
Even with tiny infrastructure, the system was resilient.
Phase 6: ChatOps and Real-Time Alerts
Monitoring was critical, but I had no modern observability stack. Instead, I built Telegram integrations:
-
Elsa workflows called the Telegram API for alerts.
-
Errors and important events went straight into chat groups.
Later, I extended it into a Telegram bot so collaborators could place orders directly in chat. Since many were constantly traveling, this fit their workflow perfectly.
From SMS to Telegram — the evolution of UX was natural and user-driven.
Phase 7: ThinCore Accounting
As transactions grew, commissions became complex. I built a ThinCore on Postgres to handle financials:
-
Commissions for collaborators.
-
Settlement with agents.
-
Payments to external partners.
It wasn’t a full ERP, but it gave the company auditable, transparent accounting for the first time.
Results After 12 Months
Within a year, the system evolved dramatically:
-
From Excel chaos to a scalable platform.
-
From manual phone calls to SMS orders, then to Telegram bots.
-
From a monolith to microservices with workflow orchestration.
-
From fragile integrations to robust API gateway + outbox events.
-
From messy roles to dedicated apps with centralized Keycloak authentication.
The system now supported nearly 1,000 collaborators, with zero significant downtime — all built by one person.