In my test environment I was making changes to the topology and I removed a pool and server from the the Enterprise Pool.
After publishing the topology it failed with the above error. To fix this we need to first find where the conference directory is homed, open up the Lync Management Shell and run the following command:
Once you have located the correct one to delete (if you make a mistake you will need to start over) then still using the same session from your previous Lync Management Shell run the following command:
- remove-csconferencedirectory -identity [number] -force
Force switch included due to errors without running it. Wait for replication to take place and then try removing the pool from your Topology and publishing it, this time it should succeed.
Hope it helps.