I always start out writing for myself, usually as a type of therapy to get me through a hard situation, or if something happened in my life that would be awesome if it was fictionalized. Eventually, as I go through many different drafts, the story begins to change form and hopefully gets better. Then I remember that if I want to get it published some day, I have to write not only for myself, but for an audience of potential readers.
Chances are, if there’s something in the story you think is cool, someone else, somewhere, will think it’s cool too. But just be sure you don’t load your story down with too many obscure references that only you and a select group of others understand. If you’re writing a fantasy novel, be sure to explain the “rules” and the magic to your readers. A story is a kind of gift that you’re giving to someone else: that they may learn from it, enjoy it, and pass it on to their friends.
You can always write stories for yourself that are kept in the privacy of your hard drive or your desk drawer. But if you’re going to submit them for publication, let others read them and determine how you can better make your story more accessible to an audience. After all, it’s excellent when someone wants to pay to read your work.