Their paper contains a proof of this, they "just" first appeal to lattice duality.
In short, to prove that lattices
$$A = B,$$
it suffices (as you say) to prove that $A\subseteq B$ and $B\subseteq A$.
What they do is use that
$$B\subseteq A\iff A^*\subseteq B^*,$$
and instead prove that $A\subseteq B$ and $A^*\subseteq B^*$.
You can verify that their proof does precisely this, but with $A = L(\cdot)$, and $B = \widehat{\alpha^\perp(\cdot)}$ your lattices.
Concretely, the containment you are missing is $\widehat{L(\cdot)}\subseteq \frac{1}{q}\alpha^\perp(\cdot)$.
Regarding this, they state
This can be seen by considering elements of $L(\cdot)$ that correspond to $s = 1$.
I haven't checked, this but I imagine they mean that $\widehat{L(\cdot)} = \{\vec t\in R^m :\forall \ell \in L(\cdot): \langle \ell, t\rangle\equiv 0\bmod q\}$.
If we replace $L(\cdot)$ in this with some subset $S\subseteq L(\cdot)$, we get a superset of $\widehat{L(\cdot)}$.
It seems they in particular state you should replace $L(\cdot)$ with the subset corresponding to the choice of $s = 1$.
Concretely, this gives us the containment.
$$\widehat{L(I_{\alpha^\times, \overline{S}}^\times)} \subseteq \{\vec t\in R^m : \forall i : (t_i\bmod q) = \alpha_i^\times\bmod I_{\overline{S}}^\times\}.$$
I don't know if this is precisely $\frac{1}{q}\alpha^\perp(\cdot)$, but their hint makes it sound like the right thing to look at.