Dewey Bunnell, the song's vocalist and writer, has said that the lyric "alligator lizards in the air" in the song is a reference to the shapes of clouds in the sky he saw in 1963 while his family was driving down the coast from Vandenberg Air Force Base near Lompoc, California where they had a flat tire. While his father changed the tire, he and his brother stood by the side of the road and watched the clouds and saw a road sign for "Ventura".
In the booklet for the boxed-set, Highway, he states that the song "reminds me of the time I lived in Omaha as a kid and how we'd walk through cornfields and chew on pieces of grass. There were cold winters, and I had images of going to California. So I think in the song I'm talking to myself, frankly: 'How long you gonna stay here, Joe?' I really believe that 'Ventura Highway' has the most lasting power of all my songs. It's not just the words — the song and the track have a certain fresh, vibrant, optimistic quality that I can still respond to". The song has a "Go West, young man" motif in the structure of a conversation between an old man named Joe and a young and hopeful kid. Joe was modeled after a "grumpy" old man he had met while his dad was stationed in Biloxi, Mississippi at Keesler Air Force Base.