Musk's time in government ended with a colourful Oval Office meeting at Trump's side.