Seems to me we leave this sort of topic to keynote speakers who almost accidentally argue for a field/issue/method/tool that they themselves (research centre, department) and associates are currently working on.
Human nature. But if people who are currently not working on defined projects/tools/applications/sites met and discussed the issues what would they say? I’ll stick my neck out and say

1 Impossible to find, access and use/re-use the models, tools, paradata.
2 No consistant, standard framework.
3 No best practices, prizes*, competitions (but plenty of surveys and state of the art papers-only they read to me more as literature reviews).
4 Interaction is not saved ( not just user data but the game mechanics and interactive tools and techniques).

How would this lead to challenges?

• Are there tools or portals that can scrape the web and auto-retrieve not just 3D models but 3D heritage models?
• Are the aims, objectives, paradata clearly available and could we create metadata wizards that coax this into the project?
• What incentives are needed to convince content creators to link to, record or even deposit their models and related assets?
• Can grant agencies (with their increased focus on data management) convince applications to deposit the models and provide ranked, hierarchical, freemium levels of access and reuse?
• Can community tools and web portals (Mediawiki, Sketchfab, Archivematica) be sharpened as kit sets for communities?

*Best of heritage? I had high hopes but I met an organiser who told me this is not primarily what Best of heritage does. It isn’t a ranking/rating/critical appraisal system but a communication of what is happening in the (museum) field.