Also not a doctor. I've personally never missed a day. However my endocrinologist has never cautioned me on taking it a day earlier or later. It's just that when you take it earlier, the next time you go to take your shot on time, the testosterone level will be lower. Meaning, I get things like more tired, kind of moody, and such before my next shot, a day or so. So if I were to take it earlier, and do it on time for the next shot, I'd have that for a longer period. But from what I know there's no real issue with doing what you did. It's best for your hormone levels and to keep from fluctuating to much and having those mood swings or other symptoms (if you get them) to do it on time. Again, not a doctor, just basing this on what I've been told by my doctors.
If your still worried just try giving your doctor a call and asking if there is any issue with taking it a day later or earlier. Again I really think theres nothing to worry about, I've heard about people missing it, and I've actually taken it two days in a row before (never missed a day, just screwed up) and there were no issues. Just don't make a habit out of screwing with your hormone levels.