Indexed columns in SQL Server do not appear to work for basic queries according to execution plan

Disclaimer: I'm not a DBA. I have picked up a few things from this board in the past that I'm building from.

I have a table of google analytics session start times. I have an index on each column. I want to filter for all sessions that were started between two dates. Screenshot below shows the query, and the index.

The query runs quickly but I do not believe it's using the index based on the Execution plan which both says that there's a missing index and shows a table scan rather than an index scan:

Why?

Is it because of something about the way I'm searching through the datetime? If instead of looking between dates, I set it equal to a date, the execution plan shows it using the index:

But it's not just this table or datetime. Here's a different table with an index on a varchar column:

And a simple query on this one also tells me I'm missing the index:

I'm stumped.

If you want SQL Server to use that index for that specific query, you need to include the session_id column. Otherwise, for each row it finds it will have to go do a key lookup into the base table. It will choose to do this for small result sets, but once you get above a certain number of rows ("the tipping point"), SQL Server thinks it's more efficient to just scan the whole base table.

You can use the INCLUDE clause of the CREATE INDEX syntax to accomplish this:

CREATE NONCLUSTERED INDEX IX_Start_Time
ON GoogleAnalytics.SessionStartTimes (session_start_time)
INCLUDE (session_id);

StackExchange Database Administrators Q#287273, answer score: 10