How do I change Postgresql to be listening on port 5432?

I've upgraded Postgresql from 9.3 to 9.6 on my machine (linux mint rafaella- so ubuntu 14.04)

As the previous version was listening on 5432 the new version is listening on 5433, but I want to change that to 5432 so that the previous configuration (rails, phppgadmin etc) work with the new postgres server.

I have changed the port in postgresql.conf from 5433 to 5432 and restarted postgres, but this didn't work:

$ sudo netstat -nltp |grep 5432     
$ sudo netstat -nltp | grep 5433
tcp        0      0 127.0.0.1:5433          0.0.0.0:*               LISTEN      25467/postgres

I also tried setting the environment variable and restarting postgres:

PGPORT=5432; export PGPORT

again, still listening on 5433 and nothing on 5432.

trying to connect gives me:

$ sudo -u postgres psql postgres 
psql: could not connect to server: No such file or directory
    Is the server running locally and accepting
    connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?

What am I doing wrong?

Output of pg_lsclusters:

$ pg_lsclusters 
Ver Cluster Port Status Owner    Data directory               Log file 
9.6 main    5432 online postgres /var/lib/postgresql/9.6/main /var/log/postgresql/postgresql-9.6-main.log

contents of pg_hba.conf:

# TYPE  DATABASE        USER            ADDRESS                 METHOD
local   all             postgres                                peer
# "local" is for Unix domain socket connections only
local   all             all                                     md5
# IPv4 local connections:
host    all             all             127.0.0.1/32            md5
# IPv6 local connections:
host    all             all             ::1/128                 md5

ypercube and deszo were able to get this user's environment up and running (per the chat transcript).

The key seemed to be running:

sudo pg_ctlcluster 9.6 main restart

but not running it as the postgres user.

See the chat log for more details.

StackExchange Database Administrators Q#185622, answer score: 2