I think would I'd do is leave the rivet....

And with a crafting knife, one of those razor sharp knives, slice cleanly thru the plastic exposing the battery... replace, then repair the plastic.. possibly by melting with a soldering iron... or simply super gluing it. - JamesJM