An interpretation for "when we two parted" by Lord Byron

I chose "When we two parted" for this week's topic because I really enjoyed reading this poem. Maybe it is because the poem was really easy to understand, but anyway I liked it. After I interpreted this poem and posted it on DTL, I searched some information about the poem and found other interpretation.

Anyway, here is my interpretation.
I thought this poem was about a couple who promised marriage each other instead of their different social rank. What made me think like this were the 21st and 22nd line, "They know not I knew thee / Who knew thee too well." So I thought maybe their relationship is in secret because of their social rank. But since one of them (I think it is the woman) is forced to engage or get married other person instead of the speaker, their relationship has to end. So I thought this poem was about the despair about losing a lover because of the social system. Well, this is what I have interpreted.

But I found some interesting fact about Byron that he had many affairs with women when he was alive. So this poem is also can be about the affair he had. 
According to Shmoop.com, the lady who is another main character of this poem is Lady Frances Wedderburn Webster. She had some affairs with several men, and Byron was one of them. It is not sure if their relationship was sexual or not, but this interpretation insists that the speaker is Byron, of course, and he is writing about his sadness in this poem.

My interpretation is quite different from the one I searched, but I really did enjoy interpreting this poem, and it is interesting how many people can interpret one single poem in various ways!