Preheat oven to 425F. Drizzle oil on sheet pan and preheat in oven until the oil is shimmering.
Meanwhile, add potatoes and baking soda to a medium pot and cover with 3 inches cold water. Salt heavily and bring to a boil. Cook until potatoes are knife tender, about 15 minutes. Drain the potatoes and let cool for 5 minutes.
Carefully spread the potatoes on the preheated baking sheet with oil (remember, the pan and oil is hot!). Spread with a spatula or wooden spoon and bake until golden brown, 30 minutes, flipping as needed to brown all sides of the potatoes (check every 10 minutes).
Meanwhile, mix the cheese, garlic, herbs, and black pepper in a large bowl.
Make the harissa mayo. In a small bowl, mix the mayo, harissa (start with a little and add more if you want more flavor/spice), and lemon juice. Season with salt and pepper.
Once the potatoes are golden brown, transfer them to the cheese mixture and toss to combine. Spread the harissa mayo on a plate, top with potatoes and serve immediately. Mayo will keep in an airtight container for up to 5 days.