We present a stringy construction of ensemble QFTs, thus realizing a disorder averaging over coupling constants. When each member of the ensemble is a CFT with a standard holographic dual, the resulting bulk can be interpreted as a single asymptotically Anti-de Sitter space geometry with multiple boundary components. This provides a UV completion of a recent proposal by Marolf and Maxfield on the existence of high-dimensional Hilbert space for baby Universes, but one which is compatible with the recently proposed Swampland considerations of McNamara and Vafa. This is possible because our construction is really an approximation which breaks down both at short distances, but also at low energies for objects with a large number of microstates. The construction thus provides an explicit set of counterexamples to various claims in the literature that holographic and effective field theory considerations can be reliably developed without an embedding in string theory.