"Republican activist Rob Eno imagined a counter-scenario: Suppose Nancy Reagan invited President Obama to debate Mitt Romney at the Ronald Reagan Presidential Library in California. Would anyone fault Obama for declining such an offer?"
Different scenario, Rob Eno leaves out the important qualification of Nancy Reagan abstaining from endorsing a candidate.
Now if Nancy Reagan agreed to that qualifier and Oblabla declined, then yes he would be faulted, and rightly so. If on the other hand Nancy Reagan took the position Vicki Kennedy did, then most likely he would not.
As it is, with 3 other debates pending, 1 declined here or there really won't make much; if any difference.