yes, basically turn small details and other brushes that shouldn't affect visibility (such as walls with windows in them, which are actually good candidates for areaportal usage instead) that aren't part of the main world geometry into func_detail. also keep in mind that any brush with at least one water face must not be turned into an entity (except func_water_analog if needed), so water will always cut visleaves.
here's what your func_details might look like (everything you see here is a func_detail, all other brushes are hidden. also don't func_detail the insides of the cells like I did in the screenshot, you can use hint/skip and areaportals to optimize them):
and here's what your world geometry would look like with everything else hidden, especially sky and displacements:
as for the stairwell, it'll look like this with func_detail hidden:
as you can see, most if not all the small walls and details and such are func_detail. nickel already mentioned the consequences of many visleaves, but doing this reduced visleaf count by about 8000 and massively reduced compile time -- 20 sec on 16 threads -- and greatly improved render performance, and of course it can be improved even further. for your sake I think you should try this at least to reduce the compile time lol