In 1982, Bathgate relocated from Dunedin to Christchurch; with the distance between him and Knox now totalling some 750 kilometers, Tall Dwarfs was relegated to a side project, with both men meeting once or twice annually to record and perform the occasional live date. The first product of their long-distance union was the EP Louis Likes His Daily Dip, issued in 1982 on the fledgling Flying Nun label; Canned Music followed a year later. With 1984's SlugBucketHairyBreathMonster, Tall Dwarfs scored their most successful record to date; the track "The Brain That Wouldn't Die" was a cult hit across New Zealand, and with its inclusion on the Flying Nun compilation Tuatara, it received significant international exposure as well. After recording 1985's That's the Short and Long of It, Bathgate moved to the U.K.; many predicted the duo's demise, but he returned after a year, and the Tall Dwarfs reunited for 1986's Throw a Sickie (so named because both Bathgate and Knox were suffering from colds during production).

With 1988's Dogma, the duo finally graduated to a proper studio; the EP featured "The Slide," a tale of euthanasia which became one of their most controversial and best-known efforts. The long out of print first four Tall Dwarfs EPs were mined for the subsequent Hello Cruel World best-of collection; the first of their records to earn proper world-wide release, it won them considerable global media coverage. In 1990 they reconvened in the studio, emerging with so much material that the longstanding EP format was finally forsaken to release a full-length LP, dubbed Weeville; a year later another album, Fork Songs, appeared. A tour of America followed in 1992, and in 1994 Tall Dwarfs issued the 3 EPs LP, a collection of 18 new tracks sequenced to resemble a trio of mini-albums. Though Knox had long maintained a solo career, Bathgate waited until 1996 to make his solo debut with Gold Lame; still, Tall Dwarfs remained intact, issuing Stumpy a year later.