I have figured out the following solution:
1. Select copy figure
2. Paste into an empty Word file.
3. Select the figure just pasted in the Word file and copy it.
(If you see this error message, you probably directly pasted Matlab figure into Gimp. You cannot directly do this, so steps 2 and 3 are essential. The transparency can be preserved in this way.)
5. Select all in the image layer of Gimp and copy it.
6. Paste into the target Word/PowerPoint file.