"We'll burn calories together," he said, but it annoyed me.

My partner and I had started a diet for a healthy lifestyle together. Everything was going well, we were supporting each other. But yesterday, he came home with a huge bag of chips and chocolate. “We can eat it together, then we’ll burn it off,” he says. I was clearly upset. Was this a joke, or had he given up on the diet? Things like this demoralize me, but I don’t know how to express it.