⚠️ Are we engineering AI systems… or just experimenting?
Over the past few years, AI and Machine Learning systems have rapidly transitioned from research prototypes to production-critical systems. However, while the capabilities of models have advanced significantly, the engineering processes surrounding them have not evolved at the same pace.
This raises an important question:
Are we truly engineering AI/ML systems — or are we still operating in an experimental paradigm?
🎯 Research Context
As part of my Ph.D. research in Software Engineering for Artificial Intelligence Systems I am investigating:
- How AI/ML systems are developed in real-world industry settings
- What Software Engineering practices are currently adopted (or missing)
- Where traditional SE processes fail to support AI/ML lifecycle needs
This research builds on:
- Prior industry experience in QA and AI-based systems
- Initial process proposals developed and evaluated with specialists
- Ongoing efforts to define a quality-oriented, agile-compatible SE process for AI/ML systems
🧩 The Problem We Are Addressing
Traditional Software Engineering processes were not designed for:
- Iterative experimentation with data and models
- Non-deterministic system behavior
- Continuous retraining and evaluation cycles
- Tight coupling between data, models, and code
As a result, teams often face challenges such as:
- Lack of standardized development workflows
- Difficulty ensuring quality and reproducibility
- Gaps between experimentation and production deployment
- Weak integration between ML lifecycle and software lifecycle
📊 Why This Survey Matters
To move beyond assumptions and isolated experiences, this study aims to collect empirical evidence from practitioners working with:
- Machine Learning / AI systems
- Software Engineering / QA / DevOps
- Data pipelines and model lifecycle management
The goal is to:
- Identify current practices and pain points
- Understand how teams actually work in practice
- Support the design of a structured, scalable, and quality-oriented AI/ML engineering process
🚀 Call to Action
If you work in any capacity with AI/ML systems, your experience is extremely valuable for this research.
👉 The survey takes approximately 20–30 minutes
👉 All responses are fully anonymous
🔗 Access the survey (CLICK HERE)
📢 Help Amplify This Research
If you know colleagues or professionals working with AI/ML systems, please consider sharing this survey.
Your network can directly contribute to advancing how AI/ML systems are engineered — making them more reliable, scalable, and aligned with Software Engineering principles.
🔭 What Comes Next
This survey is part of a broader research effort that includes:
- Controlled studies in academic environments
- Case studies in industry settings
- Iterative refinement of an AI/ML Software Engineering process
The findings will directly support the next version of this process, aiming to bridge the gap between experimentation and engineering discipline.
“Science is what we understand well enough to explain to a computer. Art is everything else we do.” - Donald Knuth