Brahms had been an active and successful choral conductor since his youth. After his move from Hamburg to Vienna, he first became known as the director of the city's Singakademie. Choral works, both accompanied and unaccompanied, form a significant part of his output, even if these works, with the exception of the German Requiem, are not heard today as frequently as they used to be.

In the late 1860s, Brahms wrote several important choral works, including the Liebeslieder-Waltzer (op. 52), the Alto Rhapsody (op. 53), the Song of Destiny (op. 54) and the Song of Triumph (op. 55). The first two of these works were inspired by Brahms's unrequited love for Julie Schumann, one of Robert and Clara Schumann's daughters. While the waltzes express the composer's ardent feelings, the Rhapsody gives voice to his disappointment, although comfort and solace appear at the end. In 1869, Julie married an Italian aristocrat, and the Alto Rhapsody was Brahms's wedding present for the young couple.  (Another personal connection:  at the first performance, the alto solo was sung by Amalie Weiss, the wife of Brahms's best friend, the great violinist Joseph Joachim.)

In this case, the title “Rhapsody” should be understood in the original Greek sense of “excerpt(s) from a larger work.” The rhapsodoi (epic singers) performed selected episodes from Homer, and it was the idea of their vocal virtuosity that carried over into the Romantic piano rhapsodies of Liszt or Brahms.

The text of the Alto Rhapsody was taken from Goethe's Winter Journey in the Harz Mountains, where a merry hunting scene in the forest is suddenly interrupted by the appearance of a misanthrope who stands outside society, suffers from his loneliness and yearns for consolation. Omitting the hunters, Brahms only depicted the protagonist's turbulent feelings in the first part of his composition.

Subsequently, the prayer for inner peace is intoned together by the soloist and the men's chorus. At this point the key changes from “tragic” C minor to “soothing” C major, and the heavenly instrument mentioned in the poem is suggested by the broken chords played by the cellos pizzicato (with plucked strings).