With the strategy and talent in place, your data science team can begin the core work of generating statistical models that predict the future or prescribe the best possible action.
Organizing your team to the right processes can lead to:
Use Case Management
Your ROI depends on tackling the use of cases with the most impact. There should be a straightforward process for identifying and prioritizing data science use cases to enable your data scientists to do high-value work.
Use cases may come from executives, operational SMEs, or data scientists themselves. The whole organization should know how to send ideas to the data science team for consideration, and the ideas should form a backlog of potential projects.
Tip: Associate a value to each use case your team develops and compare the level of effort to the benefits gained. It can be intimidating to select the single-best use case to follow, especially when multiple opinions are involved. A well-documented backlog can prove to be fruitful, though, especially following your first successful implementation.
You can use some industry-standard approaches to drive model development, like Gartner's Analytic Process, CRISP-DM, or Microsoft's Team Data Science Process. For each model, the development and tuning process should be documented in detail. Documentation should include analysis of inputs, expected outputs, different models considered, and a history of the various model design decisions and adjusted throughout development.
Building a predictive model usually requires subjective decisions. Ensuring that the data scientist makes correct assumptions and includes all relevant data is critical to creating a useful and accurate model. Each model should be independently reviewed to ensure the documentation standards are met and the model is well-designed. Some industries, like banking, even have regulatory requirements governing the independent validation of models. If you have a large data science practice, you may dedicate a team to model validation. If your team is smaller, institute a peer review system, so all models are scrutinized before considering business use.
Following the independent validation, models should be approved by a governing board that includes executive sponsors. An inventory of active and retired models should be maintained to reflect the board's decisions. Internal audit should regularly review the full validation and approval process to ensure it functions appropriately.
When the model is ready, users will want predictions to blend simply into their current reporting and operations. Where possible, levels of confidence should be reported alongside predictions to help users evaluate the accuracy of the predictions they receive and to increase adoption.
Model Monitoring & Maintenance
Models may degrade over time as new influencing factors come into play or as data shifts. They need to be reviewed periodically to help them maintain their predictive power. In addition to scheduled reviews, thresholds may be established to trigger an alert if the model inputs or outputs are abnormal. As machine learning becomes more common, well-built models can continually self-adjust over time. However, these models should still be subject to monitoring to ensure they continue to provide value. The review board should retire models that no longer serve their purpose and record those changes on the model inventory.
Written by CCG, an organization in Tampa, Florida, that helps companies become more insights-driven, solve complex challenges and accelerate growth through industry-specific data and analytics solutions.
CCG understood our project needs very well, they are very responsive and we could not ask for anything more. The solution they provided fit perfectly with our expectations and business goals.GOP Data TrustChief Data Officer
I cannot overstate the delight we experienced from the outcome of our project. I would not only recommend CCG to any company, but question why they would engage with anyone but CCG.PgiDirector of Customer Success
Working with CCG is like working with extended team members. Consultants become an integral part of the work bringing expertise for cutting edge design and development.Hillsborough County Public SchoolsChief Information and Technology Officer
CCG's team is positive and eager. They are a great big bunch of wonderful people trying to make a difference.Hillsborough County Public SchoolsDepartment Manager
I knew CCG's technical expertise and dedication to quality results would be invaluable to our project success based on our past partnerships. We could not have implemented in the short timeframe like we did without their assistance. CCG is #1 on my speed dial for successful project implementation.InCommDirector, Financial Information Systems
It was evident from the onset of negotiations through the implementation that CCG took their role in the partnership to heart and we believe it has been instrumental in our success.Interval InternationalDirector of Marketing
CCG works very hard to understand and align with our needs. It truly feels as though we are on the same team!Fortune 500 HomebuilderBI Manager
CCG came to our company in a time of much change. Their team partnered with ours, continually delivering with professionalism and efficiency. We would not be where we are today without the expertise CCG brought to the project.PSCU Financial ServicesSenior Program Manager
CCG has a good industry knowledge, we are very happy that we chose to work with CCG. They have been a great help strategically and are helping us make important decisions.Minneapolis Public SchoolsHuman Capital Coordinator
Other Vendors use the word Partnership, but CCG actually means what they say. I can’t thank them enough for their professionalism and willingness to work with us as a true Partner, not just another vendor.PODSCIO
Our CCG Consultants are total rock stars: very thorough with a solid knowledge of the financial services industry. As a bonus, they are very easy to get along with – a great fit for our team.Raymond James Financial ServicesSenior Manager of Enterprise Data
CCG's team are all amazing. Thank you, CCG, for all that you do to make us great and keep our credit unions moving forward!PSCU Financial ServicesVP Enterprise Analytics & BI
Other Vendors use the word Partnership, but CCG actually means what they say. I can’t thank them enough for their professionalism and willingness to work with us as a true Partner, not just another vendor.PODSChief Information Officer
CCG's Team is very professional and responsive. They are making our job very easy.Rollins, Inc.Senior BI Analyst
CCG did an excellent job! Their team was very flexible. They gave us everything we asked for and then some.Rooms To GoSenior BI Architect
I'm amazed at the talent at CCG, not just the skillset - they're really good people. We've already referred them once and will do so again!Ruth's Chris Hospitality GroupCIO
CCG did a great job! We're extremely impressed with what was built in a short time. CCG has delivered ahead of time and with best practices, it's been a pleasure to work with them.VologyVP of Analytics
2502 N. Rocky Point Drive, #650, Tampa, FL 33607
Phone: 813.968.3238 | Fax: 813.200.1357
8000 Avalon Blvd. Suite #100, Alpharetta, GA
Phone: 404.328.7298 | Fax: 813.200.1357