In early music, before Musical Notation and Form were clearly established, changes between meters were just at the whim of the composer. For example, Gregorian Chant was made up of notes in groups of twos and threes all over the place, usually in stepwise movement. Later on, composers began codifying rules and came up with specific key and time signatures. Composers began exploring FORM and developing frameworks on which to build their compositions.
For example, the SONATA was a work of music with a three-part structure. The basic elements of sonata form are: exposition, development, and recapitulation, in which the musical subject matter is stated, explored or expanded, and restated. There may also be an introduction, usually in slow tempo, and a coda, or tailpiece, thus expanding the FORM to 5 sections. Within this framework, the different sections change the pace and give variety at least by slowing or speeding up the tempo, but sometimes changing the keys or METERS of the different sections as well.
Variety is important in keeping an audience, or even the musicians themselves, engaged in the music. Last week I watched an old Fred Astaire / Ginger Rodgers movie "The Gay Divorcee." This was the first movie where they were intentionally paired and given star billing. But to sweeten the deal (in case they could not carry the show on their own merits), the cast was packed with colorful character actors who already had name recognition.
Although Astaire and Rodgers did dance together ("Night and Day" and a snippet of "The Continental"), they by no means had the lion's share of screen time. (I guess that back in 1934, directors were still trying to figure out a good formula for on-screen production numbers.) Anyway, "The Continental" dance went on and on for at least 15 minutes, maybe longer, with hardly any changes in meter, tonal centers, tempo, orchestration or character. The only variety was provided by who (out of the cast of hundreds) came on in different costumes to dance or sing every 32 bars. I found that it was exhausting to watch! It would have benefitted from at least throwing in a TRIO section in waltz time!
Audiences of our day appreciate VARIETY. Composers have learned to capitalize on the idea that changes in meter can really help keep the music alive for the audience. (Though modern classical composers may go a little too far sometimes by never settling on a set time signature at all.) Even in Sacred Church music, long hymn arrangements can become more interesting by sneaking in a change of meter here and there.
"Sing Praise to Him" arranged by Mack Wilberg offers a delightful look at a Sacred Hymn done with changing meter. The choir sings the traditional hymn in 4/4 first, then takes off in 7/8 meter for the rest of the verses. Very fun!