How does the computer determine whether a number is smaller or greater than another?

It might sound like a stupid question but I'm really curious to know how a computer knows that $1<2$? Also, how does a computer know that the order of integer is $1,2,3,4,5,\ldots$ and alphabet is A,B,C,D,...? Is it somewhere stored in the hardware or does the operating system provide this kind of information?

First your integer numbers are converted into binary numbers. For example, the integer 2 is converted to 0010.

The CPU uses a digital comparator:


A digital comparator or magnitude comparator is a hardware electronic
device that takes two numbers as input in binary form and determines
whether one number is greater than or less than or equal to the other
number.


Comparators are used in central processing units (CPU) and microcontrollers.


Source: https://en.wikipedia.org/wiki/Digital_comparator

In comparator hardware some gates are used (AND, OR, NAND, NOR, XOR, etc). These gates take binary inputs and give result in binary. The output can be seen from a truth table.

Inputs           Outputs
A   B     A>B    A=B    A<B
0   0     0       1      0
0   1     0       0      1
1   0     1       0      0
1   1     0       1      0

Here 0 & 1 are electronic voltages for the gate.

1 - Represents some threshold voltage which indicates some positive voltage.
0 - Represents the voltage below than the threshold.

E.g. suppose a comparator works on 5 volt (it is consideration for explanation) then:

Voltage more than 3 volt can be considered as binary-1.
Voltage below than 3 volt be considered as binary-0

If a gate gets one input as 3.5 volt and another input as 2 volt then it considers as, it takes one input as binary 1 & another input as binary 0.

These sequences of 1's & 0's are provided very fastly through the switching circuit.

The operation of a two bit digital comparator can be expressed as a truth table:

Inputs                            Outputs
   A1   A0  B1  B0  A>B    A=B   A<B        
    0   0   0   0    0      1     0
    0   0   0   1    1      0     0
    0   0   1   0    1      0     0
    0   0   1   1    1      0     0
    0   1   0   0    0      0     1
    0   1   0   1    0      1     0
    0   1   1   0    1      0     0
    0   1   1   1    1      0     0
    1   0   0   0    0      0     1
    1   0   0   1    0      0     1
    1   0   1   0    0      1     0
    1   0   1   1    1      0     0
    1   1   0   0    0      0     1
    1   1   0   1    0      0     1
    1   1   1   0    0      0     1
    1   1   1   1    0      1     0

To quote from Wikipedia:


Examples:
Consider two 4-bit binary numbers A and B such that





Here each subscript represents one of the digits in the numbers.

Equality


The binary numbers A and B will be equal if all the pairs of significant digits of both numbers are equal, i.e.,

. . .


Since the numbers are binary, the digits are either 0 or 1 and the boolean function for equality of any two digits and > can be expressed as





is 1 only if and are equal.


For the equality of A and B, all variables (for i=0,1,2,3) must be 1.
So the quality condition of A and B can be implemented using the AND operation as



The binary variable (A=B) is 1 only if all pairs of digits of the two numbers are equal.

Inequality


In order to manually determine the greater of two binary numbers, we inspect the relative magnitudes of pairs of significant digits, starting from the most significant bit, gradually proceeding towards lower significant bits until an inequality is found. When an inequality is found, if the corresponding bit of A is 1 and that of B is 0 then we conclude that A>B.
This sequential comparison can be expressed logically as:

StackExchange Computer Science Q#7074, answer score: 34