Wrap grated ginger in a dampened kitchen towel, then squeeze and twist towel firmly over a small bowl to yield scant 2 tablespoons juice.
In a medium saucepan over medium heat, combine sweetened condensed milk and whole milk. Bring to a boil, reduce heat and simmer until slightly thickened, about 3 minutes. Remove pan from heat, add chocolates and let stand 1 minute. Add butter, salt and ginger juice. Whisk until chocolates and butter melt and mixture is smooth. Pour into prepared pan. Let cool, then refrigerate until firm, about 4 hours or overnight.
To serve, lift fudge from pan using foil ends; place on a cutting board. Cut into 1-inch squares.