It's Tuesday morning. You're sitting in a conference room that smells like burnt coffee and optimism. The ops director just handed you a fifty-page PDF titled "NetSuite API Reference v2019.1" and said, "This should have everything you need." You flip to page three and realize the auth examples use a deprecated token format. Somewhere in the building, a finance clerk is manually exporting a CSV for the third time this week because "the integration broke again." Welcome to erp integration custom software — the project every stakeholder assumes is straightforward and every engineer knows is not.
I once spent four days debugging what I thought was a SOAP authentication issue. Turns out the client's SAP instance required a certificate signed by an internal CA that had been decommissioned in 2017. The "fix" involved a helpdesk ticket, two escalations, and a PDF of a scanned email from a vendor who no longer existed. That's not integration. That's digital spelunking.
ERP integration is not a coding problem. It is a political archaeology project where you dig through layers of outdated documentation, conflicting stakeholder claims, and one critical database password that lives in a retired sysadmin's inbox. The forward deployed engineer who survives is the one who maps the minefield before writing a single line of code. You don't win by being the fastest coder. You win by being the most patient cartographer.
The ERP Integration Reality Check
Most ERPs have APIs in name only. The documentation was written by someone who left the company. The sandbox environment mirrors production in the same way a movie set mirrors a real city — it looks right until you lean on a wall. You will discover that the field called customer_id in the API returns a value that maps to client_ref in the database, unless it doesn't, in which case it maps to legacy_cust_num which is only maintained by a cron job that runs on a server nobody admits to owning.
The real work of sap netsuite integration isn't writing the connector. It's finding the one person in accounting who knows which report actually contains the numbers the CFO looks at. It's discovering that the "real-time" API updates every six hours because the vendor batch-processes changes at midnight. It's realizing that the middleware your predecessor bought for fifty thousand dollars has been silently failing for eight months and the error logs are on a server that was decommissioned last quarter.
A mid-size manufacturing client I worked with had three ERPs. Not one. Three. SAP for production, NetSuite for finance, and a homegrown system from 2004 that handled shipping. None of them talked to each other. The integration project they hired me for was supposed to be "a simple sync between SAP and NetSuite." It took three weeks just to map the data fields. The shipping system used a flat file format that nobody had documented because the person who built it retired in 2011. That's the reality. That's the job.
Mapping the Minefield First
Audit what you actually have before you write a single line of integration code. The API docs are a starting point, not a source of truth. Start with the data. Export a sample. Try to import it somewhere else. See what breaks. Document every field mismatch, every encoding issue, every date format that assumes Pacific Time because the original developer lived in Portland.
Interview the people, not just the systems. The warehouse manager knows that the ERP's "available inventory" number is always wrong by Tuesday afternoon because the weekend receipts don't get processed until Wednesday morning. The finance clerk knows that the customer export drops rows when the company name contains an ampersand. The IT director knows that the API token expires every ninety days but the renewal email goes to a distribution list that includes three people who left. This is the data archaeology that makes or breaks an integration project.
Build a data map before you build a pipeline. List every source system, every destination, every field that needs to move, and every transformation required. Mark the fields you can't verify in red. Show this map to the client and watch their face as they realize how much of their operation lives in undocumented tribal knowledge. This is the moment where a good FDE earns their rate. You're not just building software. You're building a shared understanding of how the business actually works.
The Middleware Layer That Saves Sanity
Smart teams build a translation layer instead. Postgres is your friend here. Pull data from the ERP into a staging schema. Transform it. Validate it. Then push it to the destination. This adds latency, usually minutes rather than hours, but it buys you something priceless: inspectability. When the sync fails, you can query the staging table and see exactly what arrived. When the finance team disputes a number, you can trace it back to the source.
This pattern also makes testing possible. You can populate the staging table with synthetic data and verify your transforms without touching production. You can replay a failed sync by re-running the push step. You can add a new destination, such as a spreadsheet that syncs with live systems, without rewriting the extraction logic.
The alternative is point-to-point wiring, and it's a trap. Direct API-to-API integrations look elegant on a whiteboard and become nightmares in production. When the source API changes — and it will — you have to update every destination. When a new destination appears, you have to modify the source connector. When something breaks at 2 AM, you have no place to look except the production logs of a system you don't control. The translation layer is boring. It is also what keeps you employed.
When the Vendor Says No
Tactics for extracting data from systems with "no API": scheduled exports, screen scraping last resort, and the audit-trail game. Some ERP vendors treat data access like a hostage negotiation. They want annual maintenance fees. They want certified integration partners. They want you to buy their newer, more expensive platform that "has better connectivity." You don't have time for any of that.
Scheduled CSV exports are the unsung hero of legacy system integration. They're boring, they're reliable, and they work on systems that predate REST by two decades. Set up a cron job to pull the export from an FTP server, drop it into a watched folder, and process it into Postgres. It's not real-time. It doesn't need to be. Most business processes that claim to need real-time data actually need data that's accurate within a few hours. The finance team closes the books monthly. The warehouse counts inventory weekly. The CEO reads the dashboard on Mondays. Match your integration cadence to the actual business rhythm, not the theoretical ideal.
Screen scraping is the last resort, and it should feel like one. Use it only when no export exists, no API is available, and the data is critical. Document it exhaustively. Build monitoring that alerts you when the page structure changes. And start lobbying for a system upgrade the day you deploy it, because screen scraping is a bridge that collapses slowly.
Testing Integrations Without Breaking Payroll
Staging strategies, idempotency, and the rollback plan that keeps you employed. The scariest integrations are the ones that touch money. Payroll, invoicing, and inventory valuation are not places to test in production. Build a staging environment that mirrors production data without mirroring production consequences. Use synthetic customers, synthetic orders, and synthetic payments. Verify every transform with known inputs and expected outputs.
Idempotency is non-negotiable. Every sync job must be safe to run twice. If the job fails halfway through and you restart it, it should pick up where it left off without creating duplicates. This sounds obvious until you're staring at a database with three hundred identical invoice records because the retry logic wasn't idempotent. Use primary keys. Use checksums. Use update timestamps. And always, always log what you did.
The rollback plan is the document you write when you're calm and read when you're panicking. It should take thirty minutes to execute and restore the system to its pre-integration state. It should be tested. It should be stored somewhere the on-call person can find it at 3 AM without calling you. If your rollback plan requires SSH access to a server that only you have the password for, it's not a plan. It's a prayer.
Maintaining the Connection Long-Term
Monitoring, alerting, and the documentation that outlives the engagement. The day you ship the integration is not the end of the project. It's the beginning of the maintenance phase. APIs change. Tokens expire. Passwords rotate. The person who approved your access leaves and their replacement doesn't know you exist. Build monitoring that catches failure before the client does. Alert on sync latency. If the job that usually finishes by 7 AM is still running at 9, something is wrong. Alert on row counts too. If today's sync produced twelve records and yesterday produced twelve hundred, you're probably fine.
Document everything. Not just the code, but the business logic. Why does this field get transformed that way? Because the ERP stores dates as strings in MM/DD/YYYY format but the destination expects ISO 8601. Why does this sync run every four hours instead of real-time? Because the source system batch-updates at midnight and midday. Write it down. Store it in a place the client can find. The best integration is one that keeps working after you leave the client site. The second best is one that the next engineer can fix without wanting to strangle you.
ERP integration doesn't have to be a war. It just requires patience, political skill, and a willingness to dig through layers of digital sediment. The engineer who treats it like archaeology instead of coding ships faster, breaks less, and gets invited back.