Comment from Noam Brown in the article's comments section:
"I'm an author on the paper. I want to point out what I think are a few mistakes with the blog.
First, Zvi says "The strategic engine, as I evaluated it based on a sample game with six bots and a human, is mediocre at tactics and lousy at strategy." This is not the feedback we have gotten from expert Diplomacy players. The general consensus among expert Diplomacy players is that the strategy/tactics are extremely strong, perhaps expert level, but that there is room for improvement in the dialogue. I'm not familiar with Zvi's experience with the game of Diplomacy, but I did place 3rd in the North American championship this year and have learned a lot about the game of Diplomacy, including how humans play it, over the past 3 years, so I feel somewhat qualified to comment on Diplomacy strategy.
In particular, I disagree with Zvi's opinion on the bot's strategy. Zvi says "I hate France’s tactical play, both its actual plays and the communications with Russia that are based on its tactics, dating back to at least 1903. The move here to Irish Sea needs to be accompanied by a convoy of Picardy into London or Wales, fighting for Belgium here is silly."
It's well-known among experienced Diplomacy players that France needs 3 armies on its mainland in order to defend itself well from a hostile Germany. Moreover, a convoy to London or Wales is unnecessary here. By moving to Irish Sea, France is setting themselves up for the option to convoy directly into Liverpool next turn. England can't block it because they have no army on their mainland. In short, I feel pretty confident saying that France made the right move here. If Zvi still disagrees, we could pull in some consensus expert Diplomacy players to get their opinions on this.
Second, we did a 200-game tournament for no-press (that is, no-dialogue) Diplomacy back in January where players were informed that one of the players in each game was a bot. Our bot, Diplodocus, placed first in this tournament. There's a video on it here: https://www.youtube.com/watch?v=AWQFhYSD7h4&ab_channel=Diplo... . You can see in the Youtube comments that one expert Diplomacy player (Sploack) described the bot as "currently the best gunboat player in existence, or at least in the top 5." The strategy/tactics in no-press Diplomacy don't match up perfectly with full-press, but they do carry over to some extent.
Third, Zvi criticizes the fixed 1908 end date as working to the bot's advantage. The no-press tournament mentioned above did not have fixed end dates. Also, I think the fact that the full-press games ended in 1908 rather than a later date (say, 1910) actually hurt the bot. The bot handles endgame tactics quite well."
"I'm an author on the paper. I want to point out what I think are a few mistakes with the blog.
First, Zvi says "The strategic engine, as I evaluated it based on a sample game with six bots and a human, is mediocre at tactics and lousy at strategy." This is not the feedback we have gotten from expert Diplomacy players. The general consensus among expert Diplomacy players is that the strategy/tactics are extremely strong, perhaps expert level, but that there is room for improvement in the dialogue. I'm not familiar with Zvi's experience with the game of Diplomacy, but I did place 3rd in the North American championship this year and have learned a lot about the game of Diplomacy, including how humans play it, over the past 3 years, so I feel somewhat qualified to comment on Diplomacy strategy.
In particular, I disagree with Zvi's opinion on the bot's strategy. Zvi says "I hate France’s tactical play, both its actual plays and the communications with Russia that are based on its tactics, dating back to at least 1903. The move here to Irish Sea needs to be accompanied by a convoy of Picardy into London or Wales, fighting for Belgium here is silly."
It's well-known among experienced Diplomacy players that France needs 3 armies on its mainland in order to defend itself well from a hostile Germany. Moreover, a convoy to London or Wales is unnecessary here. By moving to Irish Sea, France is setting themselves up for the option to convoy directly into Liverpool next turn. England can't block it because they have no army on their mainland. In short, I feel pretty confident saying that France made the right move here. If Zvi still disagrees, we could pull in some consensus expert Diplomacy players to get their opinions on this.
Second, we did a 200-game tournament for no-press (that is, no-dialogue) Diplomacy back in January where players were informed that one of the players in each game was a bot. Our bot, Diplodocus, placed first in this tournament. There's a video on it here: https://www.youtube.com/watch?v=AWQFhYSD7h4&ab_channel=Diplo... . You can see in the Youtube comments that one expert Diplomacy player (Sploack) described the bot as "currently the best gunboat player in existence, or at least in the top 5." The strategy/tactics in no-press Diplomacy don't match up perfectly with full-press, but they do carry over to some extent.
Third, Zvi criticizes the fixed 1908 end date as working to the bot's advantage. The no-press tournament mentioned above did not have fixed end dates. Also, I think the fact that the full-press games ended in 1908 rather than a later date (say, 1910) actually hurt the bot. The bot handles endgame tactics quite well."