Preheat oven to 350F.
In a large mixing bowl, cream together the butter, granulated sugar, and brown sugar until light and fluffy. Beat in the egg, milk, and vanilla extract.
In a medium mixing bowl, whisk together the flour, instant pudding mix, cocoa powder, cornstarch, and baking soda. Pour the dry mixture into the wet mixture and mix until well combined.
Fold in the chopped Reese's minis.
Scoop the cookie dough into 24 balls and roll between your hands until smooth. Place the dough balls on the cookie sheet, leaving 2 inches between each. Flatten the top of each cookie dough ball slightly.
Bake the cookies at 350F for 7-8 minutes until set. Remove the cookies from the cookie sheet and cool completely to room temperature on a wire cooling rack.
To prepare the frosting, beat together the cream cheese and butter until light and fluffy. Add the peanut butter and milk and beat until smooth. Add the powdered sugar and mix well.
Pipe the frosting onto the bottom of 12 of the completely cooled cookies, then top with another cookie to create a cookie sandwich.