A RAG-powered, computer-vision fleet intelligence platform that cut cost leakage by 15% — built on AWS with Snap Spectacles smart glasses for hands-free inspection.
A Denver-based logistics operator running a fleet of 1,500 heavy-duty trucks was losing an estimated 15% of its operating budget to preventable “cost leakage” – undetected mechanical faults, missed maintenance windows, misfiled compliance documents, and insurance claims that could not be substantiated with evidence.
The root problem was fragmentation. Fault detection, document management, scheduling, and Engine Control Module (ECM) telematics each lived in a separate, disconnected tool. No single channel gave dispatchers, mechanics, and compliance staff a shared view of a truck’s real condition and history.
We designed and delivered a single fleet-intelligence platform on AWS that unifies all four functions. It pairs a Retrieval-Augmented Generation (RAG) knowledge assistant with computer-vision fault detection, a hands-free Snap Spectacles smart-glasses inspection workflow, an intelligent maintenance scheduler, and continuous ECM data ingestion. The same evidence pipeline also powers insurance fraud detection, giving the operator time-stamped, tamper-evident proof for every claim.
Within [X] months of rollout, the platform reduced measurable cost leakage by approximately 15% across the fleet.
The operator managed a large, aging, high-utilization fleet. Interviews with dispatch, maintenance, and compliance teams surfaced a consistent theme: there was no single channel connecting the data they each depended on. Separately, the operator was losing disputes on insurance claims because it could not produce credible, time-stamped evidence of a vehicle’s condition before and after an incident. Fraudulent and inflated claims went unchallenged.
Mechanical issues were typically caught only after a breakdown or a roadside inspection failure, driving up towing, downtime, and emergency-repair costs.
Registrations, inspection reports, driver logs, and maintenance records were scattered across email, shared drives, and paper. Retrieving the right document during an audit could take hours.
Preventive maintenance and driver assignments were planned on spreadsheets, with no link to a truck’s actual mechanical condition or ECM telemetry.
Rich engine telemetry — fault codes, fuel burn, idle time, engine load — was locked inside the trucks and rarely correlated with maintenance or cost decisions.
We built a unified fleet-intelligence platform - a genuine single pane of glass - that ingests every relevant data source and exposes it through one web console and one mobile/wearable workflow.
Mechanics and drivers capture vehicle imagery during walkarounds. A custom-trained YOLOv8 object-detection model, complemented by Amazon Rekognition Custom Labels, identifies visible faults — tire wear and damage, brake and rotor issues, fluid leaks, corrosion, and lamp/reflector defects — and grades severity automatically. Detected issues are logged against the specific vehicle and pushed straight into the scheduler.
Inspections run through Snap Spectacles smart glasses using a custom Lens built with Snap’s Camera Kit / Lens Studio SDK. Technicians keep both hands on the vehicle while the glasses stream first-person imagery to the vision pipeline and surface AR prompts and checklists in their field of view. Every capture is time-stamped and geotagged, creating a defensible inspection record without a single form being filled in by hand.
At the core is a Retrieval-Augmented Generation assistant that lets any authorized user ask natural-language questions — “When is truck 412 due for service?”, “Show the last three inspection reports for this VIN”, “Which trucks have an open DEF-system fault code?” The assistant is grounded in the client’s own documents and live ECM data, so answers are specific, cited, and auditable rather than generic.
An intelligent scheduler fuses vision findings, ECM fault codes, mileage, and compliance deadlines to generate predictive maintenance and driver-assignment plans. Instead of fixed-interval servicing, trucks are serviced when the data says they need it — cutting both premature maintenance spend and breakdown risk.
Engine Control Module data streams continuously from the fleet into the platform, where it is normalized, correlated with each vehicle’s record, and made queryable by both the scheduler and the RAG assistant — finally connecting engine reality to business decisions.
The platform is a cloud-native, event-driven system built entirely on AWS, combining managed AI services with custom-trained models. The named models and services are summarized below.
| Layer | Technology / Model |
|---|---|
| RAG generation (LLM) | Anthropic Claude 3.5 Sonnet, served via Amazon Bedrock |
| Text embeddings | Amazon Titan Text Embeddings V2 (for semantic retrieval) |
| Knowledge orchestration | Amazon Bedrock Knowledge Bases |
| Vector store | Amazon OpenSearch Serverless (vector engine) |
| Visual fault detection | Custom-trained YOLOv8 + Amazon Rekognition Custom Labels |
| Document / OCR extraction | Amazon Textract |
| Fraud & anomaly detection | Amazon Fraud Detector + custom XGBoost model on Amazon SageMaker |
| Layer | Technology / Model |
|---|---|
| Smart-glasses capture) | Snap Spectacles + Snap Camera Kit / Lens Studio SDK |
| Telemetry ingestion | AWS IoT Core → Amazon Kinesis Data Streams (ECM data) |
| Scheduling / orchestration | AWS Lambda, Amazon EventBridge, AWS Step Functions |
| Storage & data lake | Amazon S3, Amazon DynamoDB, Amazon RDS (PostgreSQL) |
| APIs & compute | Amazon API Gateway, Amazon ECS Fargate |
| Identity & access | Amazon Cognito, AWS IAM |
| Monitoring | Amazon CloudWatch, AWS X-Ray |
Because every inspection is captured through smart glasses with a time stamp, geotag, and vehicle ID, the platform produces a tamper-evident evidence trail for each truck. This same data pipeline directly strengthens insurance operations:
Adjusters can retrieve the vehicle’s documented condition immediately before and after an incident, making inflated or fabricated damage claims easy to challenge.
Amazon Fraud Detector and a custom SageMaker model score each claim against historical patterns, ECM data, and inspection history, flagging inconsistencies for human review.
Legitimate claims are substantiated instantly with cited evidence, while suspicious ones are surfaced before payment.
The unifying effect of a single channel - detection, documents, scheduling, and ECM data in one place - translated directly into recovered spend.
Fleet cost leakage
Preventable breakdowns / unplanned downtime
Inspection time per vehicle
Insurance claims substantiated with evidence
Cost leakage thrived in the gaps between disconnected tools; unifying them closed those gaps.
Servicing trucks on real ECM and vision data removed both wasted maintenance and surprise breakdowns.
Hands-free, time-stamped capture made audits and insurance disputes fast and winnable.
By replacing four disconnected tools with a single AWS-based intelligence platform – RAG, computer vision, smart-glasses capture, an intelligent scheduler, and live ECM data – the operator turned scattered data into recovered budget and defensible evidence. The result: roughly 15% less cost leakage across 1,500 trucks, faster maintenance decisions, and a fraud-resistant insurance process.