This goes without saying, but the fundamental element in a mystery story is plot. This is the engine that makes a story move, brings the characters and story to life. Without plot there would be no story structure and no story.
Plot is a plan for accomplishing an end. Or in the book How to Write Mysteries by Shannon OCork it's explained as: a situation carried through to a fitting conclusion.
To come up with a rough plot, start with your situation or what if. In the case of a mystery it could be a murder or a stolen object but usually a murder because that is the most basic heinous crime, and most murders are caused by people the victim knew.
Who had something to gain from the murder or theft? A good mystery will have more than one person who stood to gain and not all of them with good alibis. The protagonist checking each one off the list of suspects can be a scene or a chapter and sets up the plot of the story.
Add in a subplot or two and the book becomes more complex and keeps the reader guessing. According to How to Write a Mystery
every mystery has two plots- at least. The first is the real story hidden beneath the second story or apparent story. The apparent story is the murder but the real story begins to be revealed as the layers of the apparent story are peeled away a little at a time.
The layering of the story with the plot and subplot are what makes the story exciting and keeps the reader reading not only that book but any other mystery you write.
Source: How to Write Mysteries by Shannon OCork