EQtm is not mapping reducible to its complement

This is a problem from Sipser's book (marked with an asterisk).

$EQ_{TM} = \{(\langle M \rangle, \langle N \rangle)$ where $M$ and $N$ are Turing machines and $L(M) = L(N)\}$

We know that neither $EQ_{TM}$ nor $\overline{EQ_{TM}}$ are recognizable so unsure how to go about proving there can't be a mapping reduction from one to the other.

Any hints?

Suppose a computable mapping reduction $t: \Sigma^ \to \Sigma^$
from $EQ_{TM}$ to $\overline{EQ_{TM}}$ exists.

We want to create TM's $A,B$ such that for $\langle A', B' \rangle = t(\langle A,B \rangle)$, we have
\begin{align}
L(A) &= L(A') \\
L(B) &= L(B')
\end{align}

We will nest the recursion theorem to accomplish this goal. Our "inner" use of the recursion theorem will be explicit.

Let $F$ be the following TM, which ignores its input:

• Obtain own description $\langle F \rangle$ via the recursion theorem (Sipser 6.3)

• Write out the description $\langle A_0 \rangle$ of the following TM.

On input $\langle M,w \rangle$:

• Save the constant string $\langle F \rangle$

• Save $\langle M \rangle$

• Simulate step (4) and (5) of $\langle F \rangle$ to get $\langle B \rangle$

• Compute $\langle M', X' \rangle = t(\langle M, B \rangle)$

• Simulate $\langle M' \rangle$ on $w$.

• Use the recursion theorem to construct $A$ which computes $A_0(\langle A, w \rangle)$ on input $w$.

• Write out the description $\langle B_0 \rangle$ of the following TM.

On input $\langle M,w \rangle$:

• Save the constant string $\langle F \rangle$

• Save $\langle M \rangle$

• Simulate step (2) and (3) of $\langle F \rangle$ to get $\langle A \rangle B$

• Compute $\langle X', M' \rangle = t(\langle A, M \rangle)$.

• Simulate $\langle M' \rangle$ on $w$.

• Use the recursion theorem to construct $B$ which computes $B_0(\langle B, w \rangle)$ on input $w$

• print $\langle A, B \rangle$.

Note that when running the TM $A_0$ (and analogously for $B_0$), the simulation of $\langle F \rangle$ is purely "code-manipulation" of the program $\langle B_0 \rangle$: both writing out the description and constructing the recursive variant do not involve running $B_0$ at all. The latter is apparent from a close reading of the definition of $\langle R \rangle$ in Sipser, page 249. This ensures the definitions of $A$ and $B$ are not circular.

Running $F$ generates $\langle A,B \rangle$. Let $\langle A', B' \rangle = t(\langle A,B \rangle)$. Since $A$ simulates $A'$ and $B$ simulates $B'$, we have that $L(A) = L(A')$ and $L(B) = L(B')$.

It must be the case that $L(A) = L(B)$ or $L(A) \neq L(B)$. In the first case, we have
\begin{align*}
L(A) = L(B) = L(B') \neq L(A') = L(A)
\end{align*}
In the second case,
\begin{align*}
L(A) \neq L(B) = L(B') = L(A') = L(A)
\end{align*}
In either case we have elicited a contradiction of the assumption that the mapping $t$ exists.

StackExchange Computer Science Q#52444, answer score: 2