Teaching a Plane to Dance: Music-Guided Aerobatic Flight (Research Report)
Preview
So, can an airplane dance? In simulation, yes. The system picks out the strongest beats of a song, turns them into a smooth three-dimensional choreography, and a reinforcement-learning pilot learns to fly it, arriving on each beat seven times more accurately than a classical controller.
Its secret is anticipation. Like a dancer who knows what comes next, the pilot sees the upcoming movements and starts each turn early instead of catching up after the beat. Behind every move, a physics model of the aircraft checks that the flight stays within what a real aeroplane can do.
One surprising lesson came from the music itself. How sharp a turn can be depends not only on the strength of the beat and the agility of the aircraft, but also on the time between two turns. A choreography that turns on every beat leaves no time for sharp corners: like a good dancer, a good routine marks the accents rather than every count.
A huge thank you to Ruihan Jia, Hongze Wang and Prof. Dario Floreano for their guidance, and to the Laidlaw Foundation for making this internship possible!