Interesting word choice indeed. It's very literary, not something one would use in every day conversation exactly. But yeah, like you said, he's expressing regret for everything here.

He's talking not only about their relationship, but also himself. An apology to his former self of sorts, as in 'What happened to me, where did it go wrong?' 'I'm sorry my addiction won and killed that promising young writer.'

The promise they made, not questioning each other's choices - Ben can't criticize Sera for being a prostitute and Sera can't ask Ben to stop drinking - comes to a poignant end here. What if they had? Wouldn't their lives have been better?

Sera tried to change him by asking he go see a doctor but Ben rejects that idea.

I haven't seen the film since it came out, but if I remember correctly it's open-ended what happens to Sera, although we are hopeful that she will do the right thing. Ben had to die so Sera would realize that she had to change her ways, perhaps.

It's not in this clip nor in the script, but according to Wikipedia, in the film, his last word is "wow", which to me would indicate that he realizes what could have been had his alcoholism not consumed him and it's summed up beautifully in that one syllable.