How can Drupal 9's render and theme layers allow for isomorphic JavaScript?

Drupal prides itself on flexibility, and that is especially evident in the render and theme layers. Modules and themes can add #pre_render, preprocess, and a slew of other PHP functions to customize the rendered output. But there's a key piece of flexibility missing: browsers don't run PHP, so none of this code can be executed client-side.

For most of Drupal's history, this hasn't been a problem, because browser limitations and lack of JS framework capabilities and maturity have kept client-side rendering confined to niche use cases. But that's changing. What is optimal to render server-side and what is optimal to render client-side will likely continue to evolve and shake out for many years to come, but in my opinion, for Drupal to not provide any first-class support for client-side rendering as an option would be foolish.

So, let's have a conversation about what first-class support for client-side rendering might look like. Do we need to convert some PHP functions to JS to enable the option of them being executed client-side? If so, do we need Node.js to execute them server-side when desired? Are Web Components, or some other approach to componentization of the front-end, a necessary or important piece to consider? What about templates: can they and should they remain in Twig? What else should we be thinking about related to all this?

See https://www.drupal.org/node/2645666 and its related issues to get caught up on prior discussion about this problem-space.