I think this is a very complicated question for several reasons. First, we throw the words "racism" and "racist" around quite liberally. One definition is an ideology that seeks to justify or impose an unequal distribution of rights based upon racial or ethnic characteristics. This requires an existing power structure such that racism only works in one direction. An ethnic group that is being persecuted is not racist in seeking to rectify an existing inequity. In selection of romantic partners, we are typically not in an imbalanced power dynamic, although I suppose there are exceptions to this. As such, it's hard in my mind to associate one's selection (or not) with an intent to impose some unequal distribution of rights.
The second thing that makes this question complicated in my mind is the use of very simple definitions of race, which I think are much less useful in a biological sense than more specific genetic and cultural descriptors in more common use today. There is tremendous genetic diversity within what we used to commonly group in three major racial divisions. Despite this diversity, we tend to default to larger groupings, suggesting that our loose definitions of race are actually cultural description more than genetic ones. For example, children of mixed race couplings who retain some typical characteristics of blacks are often viewed in America as black, while those who don't exhibit the same characteristics may be viewed as white. The idea that a person viewed as white may be some day "revealed" to be black because of their heritage suggests that there was something bad or wrong about their blackness, which to me is a symptom of a systematic oppression, or what we call "racism."
So, I suppose having a physical attraction to people of particular genetic phenotypes or having particular physical characteristics is not in itself a necessarily racist action. (By the way, I like to think of racist as descriptive of actions, not people. Everyone, at one time or another probably commits an action that could be viewed as racist. That doesn't make someone racist, or "a" racist.) Now, if the discovery that one's intended romantic partner is from a particular racial or ethnic group different than one's original perception changes the way one feels, that may be an indication of a racist view. For example, let's say that you dated someone you thought was white, and in meeting their parents discovered that one or both were black, and that changed your intentions regarding your partner. In my mind, that would suggest something different than a preference for particular physical types. Fear of reprisal in a community with strong racial barriers may not be racist, while a judgement that the person now has some latent negative characteristic (laziness, or some such idea) is more likely an indication of a racist view.
Now we can always try to justify those choices, and there may be more or less appropriate justifications. That's why I think this is a far more complicated question than it seems on the surface. When I look at my own history, I have dated several women of African American descent, one Asian woman, several women of varied European backgrounds, a Brazilian (one might have described her as "creole," perhaps); I have married/divorced a hispanic woman and have now finally hit the jackpot with a wonderful Jewish woman of northern European heritage, so most people would probably not describe me as racist. I find beauty and attraction in all kinds and sorts of women (and men

) However, I do find myself with occasional thoughts or impressions that I catch in my mind or action as racially-based. So I believe that vilifying the idea of racism buries much of the discussion for fear of that dreaded label being applied. I try my best, and expect that others should do as well.