Preheat oven to 350°F.
Lightly grease a 5-quart baking dish (or two smaller baking dishes).
Saute red pepper, onion, and garlic in the butter until softened and set aside.
In a large mixing bowl, combine potatoes, meat, and 1 1/2 cups each of the cheeses (leaving 1 cup total to use on top).
Spread 1/2 of the potato mixture into the baking dish.
Top the potato mixture with the broccoli florets and then the red pepper mixture.
Top the vegetables with the remaining 1/2 of the potato mixture.
In the same mixing bowl, whisk together the eggs, milk, sour cream, salt and pepper.
Pour the egg mixture over the casserole.
Sprinkle with the leftover 1 cup of cheeses.
Bake, loosely covered, for 30 minutes.
Bake uncovered for 30 minutes more or until golden on top and not runny in the middle. (The dish you use and the thickness of the casserole will determine the length of baking time.) Bake until the egg is set.
Remove from the oven and let cool for 10 minutes before serving.