As with all things business, Data Migration must be valued by all the standard benchmarks of productivity, business analysis, reporting, trends, and opportunity.
We talk often, here, about Big Data and Data Transformation and how it's such an important part of a seamless, and streamlined, business operation strategy. An important and exploding use case is Data Migration.
All true – it's not new of course.
Data Migration has been around since before the first Apple Mac Plus. The difference is that today, everyone is affected: startups, small business, mom and pop shops all the way up to enterprise. Whether we're updating or upgrading a point of sale, migrating a website, changing an upstart's ecommerce platform, or setting up newsletters; somewhere along the way, those 500, 5 000, or 50 000 products, or users/subscribers, or blog articles must be kept reasonably intact for your community and client base to continue to move and grow with you.
As with all things business, Data Migration must be valued by all the standard benchmarks of productivity, business analysis, reporting, trends, and opportunity.
What is Data Migration?
Data migration is a process to move data from one place to another. Data migration may be part of a broader series of steps, integration, and automation. For the purposes of this discussion we'll focus on one-time, or infrequent Data migrations.
There will always be preparation steps in planning what is going to move, and where it is going to move to; ensuring that when your data gets to its destination, it will be appropriately usable. Some of these steps will involve Data Transformation (which we have discussed elsewhere in the site and other articles), validation, and in some cases, corrective sequestering or staging.
Show me some examples of Data Migration?
- New website: probably the easiest to visualize. You have years and years of blogs, positioning branding pages, about us numbering in the hundreds or thousands of pages and data types. Data migration can see that content show up in your new website using all your new formatting and styling.
- Moving ECommerce platforms: product is the lifeblood of your ecommerce business. Ensuring they're set up completely, accurately, including the most up to date inventory quantities and warehouse availability can save you thousands of hours, countless weeks of additional labour, as well as the reassurance of automated validation of your content along the way (humans aren't very good at this). We ran comparative calculations of automated product inventory migration versus human-managed manual data entry and unsurprisingly human performance topped out at 400 products after which point automated Data Migration became increasingly more efficient, accurate, and cost effective. Once you get past this point, a managed automated Data Migration doesn't care whether you have 400, or 40 000 product – it will diligently crunch through the scale of your data exponentially faster than human management can.
- Updating Inventory: an increasingly impactful side of product branding and positioning is conveying important changes to the supply chain as quickly and effectively as possible from manufacturer, to vendor, to wholesaler, to retailer, and anything else in between. Adopting a Data Migration plan as a key Data Transformation plan is one of the most overlooked strategic business decisions still today.
- Financial reporting: businesses across every industry have discovered and accepted that data is key to maintaining their competitive advantage. Turning their financial reporting into actionable insights has become critical to driving value and operational and competitive awareness throguhout the organization. When data migration and data transformation become central to real-time management operational models; C-level awareness can become an ongoing part of management culture, rather than batching reporting-compilation labour into 30 or 90 day increments. Competitive businesses don't wait for insight, they already know before their competition.
- Legacy upgrade: leveraging historical data for most businesses is an important data-driven value that cannot be overlooked. As always, this is fundamentally a business decision, what to keep, what to focus on, what is valuable, what is not. If the legacy platform is to remain in operation for some time instead of being retired as part of the migration project then looking into Data Migration as a real-time Data Transformation and Syndication integration may provide the best value.
Strategy, strategy, strategy
Business model and case value is at the heart of data migration. Legacy investments are too often thrown away, especially where data is involved or similarly esoteric abstraction concepts where C-level managers may not see the connection to keeping and maintaining data-as-a-business-value as part of their ongoing strategy. "Legacy", while its definition could be reinterpreted a thousand ways, could broadly be argued to mean any system, platform, or format, that is currently operational, but is likely to become less required, unsupported, or otherwise irrelevant in the foreseeable future.
As with all things business, Data Migration must be valued by all the standard benchmarks of productivity, business analysis, financial reporting and opportunity:
- reduce operating costs in technology and labour
- standardizing technology for team maintenance
- improving and making more data available to stakeholders; unlocking more value
- positioning opportunities
If this looks familiar, it should; resembling a Software Maintenance Programme plan. Starting, we have cost drivers, operational levers, and efficiency. As we move down, we gain agility, speed, opportunity, growth drivers, competitive advantages: the kind of advantages businesses realize when they unlock all the data at their disposal.
Strategy is a critical element to work through as a team before you start planning. Be sure to give each its own space! Strategy is not planning, and planning is not strategy (hint: read on to the end).
Value Analysis of Data Migration
Looking at the four key benchmarks, businesses should look at achieving Data Migration via manual methods. "What would it take to get X information re-data-entered in our new system?" If the answer exceeds 40 hours, there may already be a good case for investigating a Data Migration automation project.
Your team's training, documentation, and ongoing core competance must keep in line with the value of your systems. How difficult is it for your team to use, re-use, train new staff, and perform well with the various disparate platforms you have? There is a value to keeping productive teams actively aware of legacy platforms - are you at a point where that value is exceeding the advantages of a migration to a newer platform where your team is already well trained and up to speed?
Envisioning and unlocking new value opportunities can be one of the longer term processes in understanding the value of Data Migration and Transformation. Quite often the reason rests with the very model that the legacy platform allowed painting your business deliverables into a box. That box was likely sufficient for its time and customer expectations. That box may be limiting your vision moving forward, however, it's limitations you want to unpack and throw out in order to see beyond and what more you can do. If monthly compiled reporting was sufficient for customers before, would real time KPI access prove more powerful? If batch-compiled data required 10 person-hours of back and forther to validate an application, could fully interactive Data Migration and Transformation achieve this in 1/10th the time? Saving not only internal labour, but errors, and closing time for new accounts.
Your very value proposition can be transformed by understanding and embracing how Data Migration and Transformation affects not only internal workflow and customer service processes, but additional services you could offer, new instances where customers may find reasons to spend more money for higher and higher level value you can now deliver that you couldn't before.
Assess, Design, Build, Test, Process, and Audit
Where the data is coming from, what it contains, how it's structured, and how it must transformed to fit the new platform is key when Assessing your original data.
Design, in this case, refers to data mapping and transformation design of the solution. Is this one-time, infrequent, or real time? This is another aspect of solution design.
The Build will reflect not just the migration and transformation required and mapped, but also the robustness, performance, and timeliness of the solution requirements, by its accuracy, validation, and sheer performance and power depending on the volume involved.
Staging, or Testing, is central to gaining stakeholder alignment. By providing a staging area to monitor, validate, and re-test, you have a proving ground to understand and envision how the model will operate; and if necessary, make adjustments before committing.
Process is the carefully planned real execution of your import and transformation.
Audit, is essentially the ongoing testing of the data migration and transformation. The mission critical nature of your data or platform will dictate how important it is to ensure that you are prepared to review, assess, evaluate, and correct the quality and performance of your data migration and transformation strategy.
That box may be limiting your vision moving forward, however, it's limitations you want to unpack and throw out in order to see beyond and what more you can do
Planning, planning, planning
Be extremely clear on the how of executing your project.
Be extremely clear on who is responsible for what skillset of tasks. Are they properly equipped to be successful?
Be extremely clear on validating the data source. What amount of automated vetting is required to ensure that the source data is reliably and predictably accurate.
Be extremely clear on how errors are managed. Expect that your source data is going to cause a failure of some kind. Due to manual handling, legacy characters, missing fields, one way or the other is likely to cause some form of discontinuity, therefore you need to have a plan for how tests (see "tests" above) and ongoing validation will manage such gaps gracefully.
Understand how changes to the process, model, or adding new or changing data sources may affect your process, stakeholders, and budget. How important is security to this plan? If speed, volume, scale, or any other aspect of performance is central to success, has your team evaluated the server, platform, and technology proposed to run your process application? Is the proposed solution adequately prepared to managed the volume and flow you need to be successful?
Business Benefit and Engagement
Engaging business leaders in this process will improve the chances of smooth and predictable success. The reason is that understanding the value proposition of Data Migration and Transformation planning, systems, and scale for the business is extremely important to value what level of resources are significant for broad business objectives.
Having business leaders involved, helps to flatten decision-making not because we want to bring executive decision-making down to minutae planning, but precisely because we don't. This is what will help them scale up and away from operational data gaps, which helps to demonstrate, business operations that will be improved, best practices leaders can take away, demonstrate value propositions of more or less automation and speed. Seeing how value can be driven may spur more engagement and a richer data planning culture. Similarly, if a Data Migration and Transformation plan is not as critical, it can be scaled back.
Business value and time to realize ROI is key for leaders and all stakeholders to embrace the value proposition of a Data Migration and Transformation process, project, or solution. By understanding the scale of opportunity, whether smaller and simpler, or larger and complex, a clear perspective on value proposition is empowering for leadership to understand how much they can drive value.
