Required block property only shows up when creating the form. It doesn't show up in the edit window.

I have a drop down property that looks like this:

[Display(
    Name = "Display Mode",
    GroupName = SystemTabNames.Settings,
    Order = 20)]
[SelectOne(SelectionFactoryType = typeof(ComponentDisplayModeSelectionFactory))]
[Required]
public virtual string DisplayMode { get; set; }

When I create the block, the property shows up. But after the block has been created, it doesn't show up when I edit the block. This is in my local dev enviornment. It hasn't been published yet.