AI was once heralded as the next big leap for data-driven organizations, but integrating it into existing data frameworks is proving more challenging than anticipated.
AI doesn’t just produce and handle vast amounts of data; it also introduces untraceable biases in training datasets, which can result in biased models that affect both performance and compliance. Moreover, data specialists aiming to enhance data quality and maintain its integrity are venturing into relatively uncharted territory due to AI’s complexities.
Techopedia explores the intricacies of when it’s beneficial to merge AI with data operations and when it’s best to maintain separation.
Widespread AI Adoption with Reservations
The “2024 IT Trends Report, AI: Friend or Foe?” by SolarWinds revealed that while a vast majority (88%) of organizations are adopting or planning to adopt AI, only a few are confident in their readiness to integrate this technology. The primary hurdles include data management, infrastructure robustness, and security concerns.
In April, Gartner indicated that 69% of organizations feel compelled to evolve or completely rethink their data and analytics (D&A) strategies due to AI’s disruptive impact. Additionally, about 38% foresee a comprehensive overhaul of their D&A architecture within the next year and a half.
AI Data Management: Comprehensive Tracking from Start to Finish
Often, data-driven companies may not prioritize tracking data flows until it becomes a pressing issue. Adding AI to the mix exacerbates this problem, transforming it into a significant challenge.
Bakul Banthia, Co-Founder of Tessell, a cloud database as a service (DBaaS) provider, shared insights with Techopedia:
“Data flow should be managed much like energy or water—flowing from a central hub to various nodes and finally reaching end-users. As long as access to data is controlled and its reach is well-defined, data flows can be effectively managed.”
However, Banthia cautioned that this task is daunting and requires rigorous discipline. He pointed out that breaches often occur in secondary environments, which may be neglected from a security and policy standpoint. He emphasized that data leaders must establish and enforce a unified policy covering all data users, whether human or machine.
John Stringer, Head of Product at Next DLP, a data protection company, also weighed in on the importance of understanding data’s origin, pathway, and endpoint to safeguard sensitive information.
Challenges Posed by AI to Traditional Data Tracking Techniques
The dynamic nature of AI data flows, characterized by complex transformations and manipulations, poses challenges for traditional data quality monitoring tools. These tools often struggle to track and maintain visibility over the sophisticated, non-linear data pathways common in AI operations.
If these data quality issues go unnoticed, they can be exacerbated by AI systems, leading to outcomes that are biased or inaccurate.
Stringer highlighted that traditional data loss prevention (DLP) solutions are typically not equipped to handle the complexities introduced by AI, particularly generative AI that creates vast new data types.
He noted, “Innovative platforms are now enhancing visibility and control from data creation to deletion, which is critical for managing the intricacies of modern data, including those generated by AI technologies.”
Real-World Insights: Integrating Data and AI
Bob Rogers, former Chief Data Scientist at Intel and CEO of Oii.ai, shared his experiences with Techopedia. He discussed leading a team at UCSF that developed the world’s first FDA-cleared AI for X-ray devices, now known as the GE Critical Care Suite. The complex data processing pipelines they developed were essential for the consistent, repeatable outcomes necessary for FDA approval.
Rogers emphasized the importance of meticulous data lifecycle management, from collection and annotation to transformation, to ensure successful AI integration and regulatory compliance.
The Bottom Line
The journey to integrating AI into data-driven frameworks presents new challenges, from managing complex workflows to addressing vast data consumption and the inherent limitations of traditional data tracking tools. Fortunately, emerging solutions are providing the visibility and control needed to manage these complexities effectively, enhancing both data lineage and security across the AI supply chain.
Source: techopedia.com
