Elle
When I read your post, the first thing I thought of was the ‘acting and talking black or white’ issue because your question is along the same lines. First, let me clarify your choice of words: you are not wearing ‘black clothes.’ You are wearing clothes created by a designer who happens to be African American. There is nothing wrong with you wearing clothes by African-American designers. Those in the dominant group (whites) have been appropriating aspects that are attributed to the African-American culture since forever. However, there is something wrong with you wearing the clothing if you are trying to be someone other than yourself. Clothing is nothing more than self-expression.
I don’t know why some African Americans would have a problem with you wearing clothes created by African Americans, unless they ignorantly perceive your self-expression as misappropriating the African-American culture or not being yourself.
Lastly, I would refrain from using the term ‘ghetto’ to describe clothing or anything else you may attribute to African-American culture. It is a pejorative term and considered offensive to those who live in public housing. I encourage you to look up the word and how it relates to the Jewish culture.