Once it opens, files that are available offline will display a small icon next to them that looks like a checkmark inside a circle. If you hover over this icon it will say Available offline.