This project overhauls velocity-based training by switching legacy IoT-based methods with an AI-powered mobile app. The app provides real-time feedback on barbell exercises with metrics like rep counts, one-rep max (1RM), barbell velocity, and training stress score (TSS). Employing advanced AI and CoreML, the app guarantees precision, affordability, and accessibility.
A man lifting barbell, a part of velocity-based strength training
Introduction
Project Aim: Give athletes an AI-driven fitness solution to track barbell exercises without external hardware.
Key Challenges: Traditional solutions were too expensive and IoT-dependent. This app eliminates those reliances and provides precise performance metrics via AI.
Features:
Real-time rep accuracy.
Custom ML model for barbell velocity monitoring.
Dynamic workout visualizations.
Primary Goals
Create a Hardware-Free, AI-Based Velocity Tracking Solution
Why: Traditional velocity tracking systems rely on specialized hardware like sensors, wearables, or external devices to collect data. These solutions are expensive, require maintenance, and are unsuitable for all users or environments. On the other hand, a hardware-free solution uses AI and the built-in capabilities of a smartphone (camera and accelerometers) to track velocity without requiring extra devices.
Challenges: To track velocity without hardware, the system needs to extract motion data from the video or sensor data provided by the phone’s built-in features. The AI needs to analyze video frames, discern movement patterns, and estimate velocity in real-time, all while considering challenges like changing lighting conditions, varying distances, and different movement speeds.
Why: IoT devices, like motion sensors and accelerometers, are used for real-time velocity tracking because they provide high accuracy with low latency. Real-time accuracy like these devices is essential for the AI solution to be effective, especially in use cases like sports, fitness, or performance tracking, where real-time feedback is crucial.
Challenges: Real-time processing is not limited to only detecting and tracking movement but involves providing feedback to the user. This means the system needs to handle high computational demands while maintaining accuracy and no delays or errors in tracking. Factors like FPS variations, motion blur, and device limitations (e.g., camera quality) must be considered to ensure consistency.
Build Compatibility Across All iPhone Variants
Why: The solution should work on all iPhone models so users with different devices can use the app. Also, variations in hardware (e.g., camera quality, processing power, and sensors) can affect AI algorithms. Being compatible with all devices maximizes the app’s reach and user base.
Challenges: Older iPhone models often have less powerful processors, lower camera resolution, or different sensor configurations, affecting the app’s ability to track velocity. The AI model must be adaptive to handle these differences and provide accurate results.
Provide a Great User Experience with Meaningful Insights
Why: While precise velocity tracking is essential, the main objective is to deliver a valuable and user-friendly experience. The system should monitor velocity and analyze and present the data in a relevant and useful way for all users, whether athletes, coaches, or everyday users.
Challenges: To provide actionable insights, the AI system must examine the velocity data and turn it into usable feedback. This could involve highlighting performance trends, suggesting areas for improvement, or showing progress over time. Moreover, the user interface (UI) should be user-friendly, requiring little setup or complexity, so users can effortlessly engage with the app and grasp the results.
Challenges and Solutions
1. Inconsistent Barbell Detection in Gym Environment:
Gym environments present a notable challenge due to the variety and quantity of available equipment. Barbells often resemble other gym equipment, such as dumbbells, weight plates, and kettlebells. Moreover, the background clutter in the gym environments, like mirrors, equipment racks, and uneven lighting, deterred the model’s ability to consistently detect and track the correct barbell.
Causes:
Visual Overlap: Barbells are often visually identical to other gym equipment, especially in crowded or complex setups.
Lighting Variability: Shadows, mirror glares, and uneven lighting made the barbell look different across frames.
Dynamic Environment: Moving people and equipment added noise, thus interfering with tracking.
Solutions:
Added more diversity to the dataset with different gym environments involving varied lighting conditions, equipment types, and user scenarios.
A proximity-based detection logic prioritizes the barbell closest to the user’s position.
Trained ML models with additional labels for barbell rods–to differentiate them from weight plates and other objects.
2. Frame per second (FPS) Variations in Real-Time Velocity:
FPS variations across various iPhone models affected real-time velocity calculations. The reason is that high-end devices had a steady FPS but older devices like the iPhone SE and iPhone 7 show a drop in frame rates during intense computations resulting in inaccurate velocity readings.
Causes:
Device Hardware Constraints: Older devices had less computational power and strumbled with real-time ML model inferences.
Frame Drops: Lower FPS resulted in longer time intervals between frames, affecting the precision of displacement and velocity calculations.
Synchronization Issues: FPS variations affected the consistency of the time stamps in the velocity formula.
Solutions:
Incorporated adaptive algorithms that adjust time intervals in velocity calculations based on the device’s FPS.
Preprocessed the video frames to reduce computational load on older devices and maintain smoothness.
Optimized the ML model to reduce its size and complexity without sacrificing accuracy to infer faster on low-end devices.
3. False Positives from Circular Objects Similar to Barbell Plates:
During initial testing, the model would misclassify circular objects like gym mats, stability balls, or even nearby weight plates as barbells. This affected the tracking accuracy as the model would calculate metrics based on irrelevant objects.
Causes:
Similar Shapes: Barbell plates have circular shapes similar to many other gym objects, making them hard to detect.
Dataset Bias: The initial dataset had images lacking differentiation between barbells and circular objects.
Overfitting: Previous iterations of the ML model were too focused on circular features and overlooked contextual details like the barbell rod.
Solutions:
The model was retrained with a much larger dataset that included false positive scenarios (e.g., gym mats, stability balls) marked as non-barbells.
Contextual features were incorporated into the model training process, such as recognizing the presence of barbell rods to enhance circular plate detection.
Implemented Solutions:
1. Custom Barbell Detection Model:
A custom barbell detection model was created and trained with a diverse dataset of over 15,000 images to detect barbell plates and rods in different gym environments. This addressed the false positives and inconsistent detection by improving the model’s understanding of barbell features.
Model Features:
Dataset Diversity: The training dataset encompassed a variety of scenarios like different lighting, gym layouts, and types of equipment.
Labeling Strategy: Added labels for barbell rods and plates separately so the model can differentiate between the two and reduce false positives from similar circular objects.
Augmentation: To boost the model’s ability to generalize, the dataset was augmented using techniques like random cropping, rotation, brightness adjustment, and blurring to simulate real-world conditions.
Implementation Steps:
Data Collection: Images of barbells were taken from various angles and environments. Scenarios that could lead to false positives, such as stability balls and gym mats, were explicitly included.
Model Training: The dataset was fed into a deep learning framework using YOLO-based architecture for real-time object detection.
Testing and Iteration: The model was fine-tuned by testing in a controlled environment and addressing detection inconsistencies.
Results:
Accuracy: The model achieved 90–95% detection accuracy and reduced misclassification.
Performance: Works on devices with limited computational power.
AI-powered barbell tracking technology detecting and analyzing movements
2. Proximity Logic:
To handle multiple barbells or similar objects in a gym environment, proximity logic was added. This ensures the app prioritizes the detection of the barbell closest to the user’s position, which is in line with the movement.
How it works:
Distance Calculation: The detected barbells are prioritized based on their Euclidean distance from the center of the user’s activity zone.
Dynamic Filtering: Only the closest barbell that meets the movement threshold is tracked for velocity and reps.
Solutions:
Removed distractions from other gym equipment or multiple barbells in the frame.
Decreased tracking errors in crowded gym environments.
Results:
Better Detection Accuracy: Concentrated solely on the relevant barbell, reducing false positives.
Enhanced User Experience: Consistent tracking in a dynamic environment.
3. Velocity Calculation Optimization:
Velocity calculation is a key feature that gives athletes valuable insights into their performance. An algorithm was created to function reliably across devices with varying FPS to ensure high accuracy.
Algorithm:
Frame-to-Frame Displacement: Calculates the barbell’s movement between consecutive frames by using its coordinates from the detection model.
Real-Time Adaptation: Adaptive algorithms were integrated to adjust for FPS variations, ensuring consistent performance across multiple devices.
Smoothing: Moving averages were used to smooth out minor fluctuations due to frame drops or detection noise.
Testing:
Compared performance with leading IoT devices to ensure velocity calculation was within 5-10% variance.
Multiple iPhone models, from the iPhone SE to the iPhone 16, were tested to enhance FPS handling for each.
Results:
Accuracy: Velocity metrics were as precise as specialized IoT devices.
Compatibility: The system worked effectively on multiple iPhone models.
AI-driven mobile app doing real-time velocity tracking
Technical Implementation
Barbell Tracking
Description We developed a custom machine learning (ML) model to effectively detect and track barbell movements. This innovation removed the necessity for hardware markers or manual calibration, making it more convenient for users.
Implementation Details
Barbell Detection: The ML model was trained on 15,000+ images to detect barbell plates and rods.
Movement Tracking: Barbell coordinates were calculated across frames to track the movement.
Real-Time Feedback: Users got real-time visualization of the barbell’s path, including its range of motion and velocity.
Solutions
We tackled the challenges of detecting barbells in cluttered gym settings.
We also removed false positives from similar objects like weights and circular gym equipment.
Results
Accuracy: Attained a detection accuracy of 90-95%.
Convenience: Eliminated the need for external hardware or IoT devices, thus providing a seamless user experience.
Our AI model detects barbells accurately in a cluttered gym setting.
Velocity Metrics
Description Velocity is a key metric for velocity-based training. The app determines velocity by measuring the barbell’s displacement between consecutive frames and the time interval between those frames.
Implementation Details
Displacement: This involves measuring the distance between the barbell’s coordinates in consecutive frames.
Time Interval: Calculated the time difference between frames and adjusted for FPS.
Velocity Calculation: Calculated by combining displacement and time interval.
Data Smoothing: Applied moving averages to reduce noise and stabilize readings.
Solutions
Addressed the impact of FPS variations on velocity calculations.
Provided reliable metrics even on older devices with lower frame rates.
Results
Accuracy: Delivered velocity metrics with a 5-10% variance compared to top IoT devices.
Insights: Users could analyze performance in real time with actionable feedback.
Our AI-powered barbell tracking app detecting and analyzing reps, ROM, and velocity
Range of Motion (ROM)
Description ROM metrics were utilized to validate rep counts and movement quality. Thresholds for upper and lower motion limits were set to ensure precise reps tracking.
Implementation Details
Repetition Validation: The reps were validated and were counted as “reps” only when full ROM was achieved.
Movement Quality: The smoothness and consistency of the movements were assessed, offering insights into technique and areas needing improvement.
Solutions
Removed false rep counts from incomplete movements.
Encouraged proper form and full range of motion to maximize workout efficiency.
Results
User Experience: Provided accurate repetition counts, enhancing the precision of workout tracking.
Device Compatibility
Description The app has been fine-tuned for compatibility and performance across various iPhone models, including older ones like the iPhone SE and iPhone 7.
Implementation Details
Adaptive Algorithms: Modified algorithms to suit lower processing power and frame rates of older models.
Lightweight Models: TensorFlow Lite and CoreML models were used to reduce computational load.
Memory Management: Streamlined resource usage to avoid crashes and provide a smooth experience on devices with limited RAM.
Solutions
Facilitated real-time tracking on devices with different hardware capabilities.
Maintained a consistent user experience across all supported iPhone models.
Results
Compatibility: Guaranteed reliable performance on older devices, expanding the app’s accessibility.
Efficiency: Achieved a balanced computational efficiency and tracking accuracy for seamless operation.
Notable Outcomes
1. Accuracy:
The app showed impressive accuracy in detecting and tracking barbell movements, achieving a 90-95% precision rate. This level of performance was on par with top IoT-based solutions that typically depend on hardware for their accuracy.
Barbell Detection:
The custom ML model was trained on over 15,000 barbell images, capturing different angles, lighting conditions, and gym setups. This comprehensive training enabled consistent detection, even in cluttered settings.
Additional labeling, such as for barbell rods, removed false positives from objects like circular gym weights or mirrors.
Proximity-based logic integration ensured only the nearest barbell was tracked, improving reliability in busy gym environments.
Velocity Tracking:
Velocity calculations were fine-tuned using the displacement between consecutive frames and time intervals, providing real-time feedback with minimal variance compared to IoT devices.
Data smoothing methods, i.e., e.g. moving averages, improved the precision of velocity metrics, guaranteeing reliable results.
2. Performance Metrics:
The app consistently provided high-quality performance metrics with minimal variance compared to non-IoT solutions.
Velocity Data:
Variance in velocity was 5-10% compared to industry-standard IoT devices like FLEX and Vitruve.
Adaptive algorithms adjusted for device FPS variations, ensuring consistent results across multiple iPhone models.
Real-time barbell tracking remained smooth and accurate, even during dynamic movements like powerlifting and snatch lifts.
Range of Motion (ROM):
Accurate upper and lower movement limit thresholds ensured only complete repetitions were counted.
ROM metrics validated movement quality, promoting proper form and full-range exercises, directly contributing to better training outcomes.
3. User Experience:
The app offers a seamless and simple user experience, featuring tools designed for athletes and fitness enthusiasts.
Tracking:
Athletes could track reps, velocity, and ROM without IoT devices or manual intervention.
Real-Time Analytics:
Users receive instant feedback on their performance metrics, which allows them to adjust their training in real time.
UI was designed to present data in a visually appealing and easily digestible format, donned with graphs and progress summaries.
Athlete Feedback:
Initial user testing yielded positive responses; athletes loved the app’s accuracy and elimination of IoT devices.
The app was simple and effective for both professional athletes and casual gym-goers.
4. Device Compatibility:
The app is optimized for a range of iPhone models, from older versions like the iPhone SE and iPhone 7 to newer ones like the iPhone 16.
FPS Optimization:
The app features dynamic frame rate handling to ensure smooth operation on devices with different processing power and camera capabilities.
On older devices, frames are processed selectively to balance performance and accuracy, while newer devices use higher FPS for more accurate tracking.
Processing Load Management:
TensorFlow Lite and CoreML models were optimized to consume fewer resources so low-end devices could handle real-time barbell tracking without lag.
Memory management strategies prevented crashes and ensured the app’s seamless operation on all supported devices.
Testing:
The app was tested on multiple iPhone models for consistent results across all devices.
The app offered a uniform experience, promising consistency irrespective of the hardware.
Overall Impact:
The results of these efforts positioned the app as a solid and highly accessible solution for velocity training. By providing IoT-level accuracy without extra hardware, the app performed well and set a new standard for AI-driven fitness tracking.
If you have a similar software idea that leverages AI, consult our expert AI developers to guide you and help transform your vision into a real solution.
Graphical comparison of our demo app’s performance with IoT-based devices–Vitruve and Flex
Lessons Learned
Data Diversity in Training Datasets is a key ML Model Robustness
Why: A diverse dataset is crucial for an ML model to perform well across diverse scenarios, environments, and applications. In your project, whether the ML model is focused on analyzing movement, detecting objects, or serving another function, it must be exposed to various conditions to build its robustness.
Problems: Training a model with a narrow dataset can lead to overfitting, where the model performs well on known data but struggles with real-world variability. For instance, if a movement detection algorithm is trained solely on clear footage from well-lit gym environments, it may not accurately detect movement in dimly lit, crowded, or disorganized settings.
Solution: To solve this problem, it is crucial to incorporate diverse examples in your dataset, such as variations in lighting, angles, cluttered spaces, different types of motion, and user behavior. For example, including footage from gyms with varying sizes of crowds, individuals of various body types, and a range of lighting scenarios will better prepare the model for real-world challenges.
Techniques like data augmentation—such as adding noise, modifying lighting, or using mirrored images—can also help increase dataset diversity.
Result: This approach leads to a more resilient ML model capable of effectively managing various environments and conditions, ensuring consistent and reliable performance in practical applications.
Real-Time FPS Variations Require Adaptive Algorithms for Reliable Outcomes
Why: In scenarios where real-time processing is essential (e.g., movement tracking or video processing), it’s crucial to maintain steady frames per second (FPS) to ensure smooth and accurate results. However, factors like hardware constraints, processing capabilities, or network conditions can cause FPS to fluctuate.
Problems: Unexpected drops in FPS can result in missed frames, leading to inaccurate analysis and reduced model performance. In certain situations, losing frames might cause the system to lose track of an object or fail to notice environmental changes, resulting in unreliable outcomes.
For instance, in a fitness application that monitors exercise movements, dropped frames could overlook essential aspects of a user’s motion, resulting in incorrect feedback or progress tracking.
Solution: To address this problem, implementing adaptive algorithms that can adapt to varying FPS is crucial. This could include:
Temporal Smoothing: Using temporal filters to smooth out the erratic behavior caused by dropped frames so the model can make predictions based on available data.
Frame Prediction: Using interpolation or motion estimation to predict missing frames, ensuring that analysis remains uninterrupted even during temporary FPS drops.
Dynamic Adjustment: Adjusting the model’s complexity or processing load based on available FPS. For example, if FPS decreases, the algorithm could temporarily reduce the computational load to concentrate on essential features, enhancing overall performance.
Result: The system is more robust to FPS variations and works consistently under different hardware and environmental conditions.
Proximity-Based Logic is Key for Cluttered Environments Like Gyms
Why: In environments with a high density of people, equipment, and other distractions (e.g., gyms), proximity-based logic is essential to make the system focus on the most relevant objects or individuals. This logic helps the system distinguish between foreground and background, thus improving tracking accuracy and reducing errors.
Problems: In a gym setting, there are many overlapping objects, people moving in different directions or equipment that blocks the view. Without proximity-based logic, the system can track irrelevant objects or lose track of the intended subject (e.g., a person lifting weights or doing an exercise).
Solution: Implement proximity-based algorithms prioritizing nearby objects or individuals over distant ones. For example, the system can use depth sensors or computer vision to calculate the distance of objects and focus on the ones within a specific range. Also, motion-based proximity filters can help the system identify and track active users and ignore the background activity.
Result: The model works better in cluttered environments, improves object tracking, and gives a better user experience by focusing on relevant data and reducing distractions. This is especially important in gyms where users move close to each other or equipment.
Future Directions
Enhance tracking features to encompass dumbbells and kettlebells.
Personalized performance recommendations based on tracked metrics.
Improve FPS for older devices.
Integrate with wearables for more data insights.
Space-O technologies (AI) strives to build solutions that revolutionize the world of fitness and healthcare. Our expert AI developers are continuously working on innovating AI-driven healthcare technologies to make fitness and healthcare more accessible and affordable.
Conclusion
This AI-driven app showcases how cutting-edge machine learning and CoreML can make complex techniques, such as velocity-based training, more straightforward. By removing the need for specialized hardware, the app guarantees accessibility, cost-effectiveness, and accurate tracking for athletes around the globe.
Looking To Build an AI-Powered Fitness App?
Contact us. Our expert AI developers can assist you in building a robust AI-powered fitness app using top-notch AI models that are within your budget.
Last Updated: January, 31 2025
