Start by cooking the bacon in a large skillet over medium heat until it's crispy and golden, about 8-10 minutes, stirring occasionally. While the bacon cooks, place the potatoes in a large pot of cold water with 2.5 teaspoons of salt. Bring to a boil, then reduce heat and simmer for 12-15 minutes until the potatoes are just tender when pierced with a fork but still hold their shape—you want them firm enough to absorb dressing without falling apart. Once the bacon is done, transfer it to a paper towel-lined plate to drain and cool, then crumble it into small pieces. Drain the hot potatoes thoroughly in a colander.
While the potatoes are still warm, prepare the vinaigrette by whisking together the Dijon mustard, white wine vinegar, minced garlic, sugar, 3/4 teaspoon salt, and black pepper in a small bowl. Slowly whisk in the olive oil until emulsified, then add the water to lighten it slightly. Pour this warm dressing over the drained potatoes and gently toss to coat evenly. Let the potatoes sit and absorb the dressing for at least 30 minutes at room temperature—this is where the real flavor develops, so don't skip this step. I like to use Yukon Gold potatoes because they're naturally creamy and absorb dressing beautifully without becoming mushy.
While the potatoes are absorbing the vinaigrette, prepare all your vegetables by dicing the cucumber into 1/2-inch cubes, chopping the celery into small pieces, dicing the white onion finely, and chopping the fresh chives. Keep these in separate bowls so they're ready to combine when needed. I like to keep the cucumber separate until the very end—it stays crispier and doesn't weep water into the salad if you wait to add it just before serving.
In a medium bowl, whisk together the mayonnaise, sour cream, and French dressing until smooth and fully combined. Stir in the finely grated horseradish, the remaining 3/4 teaspoon salt, and 1/2 teaspoon black pepper. Taste and adjust seasoning as needed—the horseradish should give a pleasant peppery kick without being overwhelming. This creamy dressing acts as the binding agent for the final salad, so make sure it's well-seasoned before combining.
Add the creamy dressing from Step 4 to the vinaigrette-coated potatoes along with the celery, onion, and most of the crumbled bacon from Step 1, reserving about 1/4 of the bacon for garnish. Fold everything together gently until just combined, being careful not to break up the potatoes. If you're serving immediately, add the diced cucumber now; if you're making this ahead, add it just before serving to keep it crisp. Transfer to a serving bowl and top with the reserved bacon and fresh chives. For best flavor, let the finished salad rest for 15-20 minutes before serving so all the flavors meld together.