CARLA environments, image and vehicle-state fusion, actor–critic training and checkpoint recovery.
Public simulation work; physical deployment is documented separately in the golf-cart project.
Technical deep dive Architecture, implementation & tradeoffs
Demonstrations become control features.
- 01
Observe
RGB + additional channelsFive-channel input representation paired with expert controls.
- 02
Imitate
visual features → control headsTransfer RGB features and train steering, throttle and brake predictions.
- 03
Reuse
checkpoint feature extractorCarry the learned visual representation into an interaction-based agent.
- 04
Improve
visual features + vehicle stateCondition SAC on perception features and low-dimensional vehicle state.
A checkpoint is also a reusable representation. The next experiment asks what interaction can add to features first learned from expert driving.
- Learning stages
- Supervised imitation → representation reuse → actor–critic
- Control
- Steering, throttle and brake predictions
- Temporal scope
- The current recurrent path receives a single timestep
Observation to control
- Adapted the visual model to five input channels while retaining pretrained RGB weights for the first three.
- Used separate steering, throttle and brake heads for the imitation path.
- Built the CARLA environment interface and the observation/action plumbing for interactive training.
Representation reuse
- Loaded the imitation checkpoint into a custom feature extractor and removed the final control-regression heads.
- Combined the extracted visual representation with state inputs through a learned projection.
- Integrated the extractor with Stable-Baselines3 SAC and separate policy/critic networks.
The experiment infrastructure
- Implemented checkpoint discovery and resume logic, evaluation and monitoring utilities.
- Added multi-environment training paths and tools for recovering interrupted experiments.
- Kept collected demonstrations, imitation models and interactive-learning experiments in the same development trail.
Private control extensions
- Extended the autonomous-driving work with MPCC comparison, modified PPO/SAC and a multi-head attention encoder.
- Used the same methods in the physical golf-cart work, including an RL adapter and industrial-control integration. These extensions are maintained separately from the public CARLA repository.
Transfer the representation; inspect the control problem again.
A representation learned from expert examples supplies a useful starting point, but interactive training changes the observations the agent encounters. The pipeline makes the feature-transfer boundary explicit so the demonstration model, state inputs and actor–critic objective can be inspected separately.
CARLA simulation work with separate private MPCC, modified PPO/SAC and attention-encoder extensions. The related physical golf-cart case study covers the vehicle deployment.