That's because empty RAM is wasted RAM.
You want to keep RAM full of stuff you aren't using right now. That stuff can be erased almost instantly if you need the RAM for something else. That's why OSs list "Available RAM" and "Free RAM" as two separate things.
Removing all the idle apps from RAM just means you need to wait longer every time you open one of them.
High resolution graphical assets are big. That's why we're now getting 32GB video cards in desktop PCs.
Sure, nobody's doing super high end graphics on a phone, but it doesn't take much high resolution art to take a big bite out of that 6GB. Phones don't have separate system and video RAM, remember.