A black Indiana congressman used a lynching metaphor to describe tea party policies he says would turn minorities into "second class citizens," and the lone Republican member of the Congressional Black Caucus threatened Wednesday to quit the group in protest.

Democrat Andre Carson of Indianapolis made the remarks to a crowd at a black caucus-sponsored event in Miami, arguing that some tea party politicians are trying to block the economic advancement of blacks and other minorities.

"Some of them in Congress right now with this tea party movement would love to see you and me — hanging on a tree," the Indiana congressman said.

The comments from Carson's speech last week were first posted online Tuesday by the Glenn Beck-founded website The Blaze.

Allen West of Florida, the only Republican member of the black caucus, said on the program "Fox & Friends" that he might quit the panel over what he said were "reprehensible" comments.

Carson told The Associated Press on Wednesday that he and West have a "cordial" relationship but that nobody on the black caucus would "lose sleep" if West left the panel because of the comments. He said the Democrats who make up the rest of the caucus have supported him.

Carson said that while he wishes he would have chosen different analogies in his speech, he would not have changed its substance. He said the language he used has become a "distraction" to the message he was trying to get across about some tea party members.

"I'm deeply concerned about some of the extremist elements who I feel have been a distraction to many of the well-meaning Americans who affiliate themselves with the tea party," Carson said. "What I was referencing was is there are some in the tea party that want to take America back to a time when certain minority groups and women had greater economic struggles and fewer opportunities than they do now."

Indiana tea party organizers said Carson owes them an apology and questioned whether Carson was trying to "incite a riot," The Indianapolis Star reported.

"He owes every Hoosier an apology for his comment," Monica Boyer, a Kosciusko County tea party leader, told the newspaper. "Since Mr. Carson can't stand on substance or truth, the only thing he has left is to play the race card."

State Rep. Vanessa Summers, an Indianapolis Democrat who is chairwoman of the Indiana Black Legislative Caucus, said Carson shouldn't apologize for expressing what many people feel.

She said politicians in Indiana and nationally have pushed tea party-supported legislation that would cut spending on programs important to the minority community, including education.

"Some of that affected some of us in a very detrimental way," Summers said. "The kind of agenda and the kind of legislation they are pushing is harmful to people of my constituency who happen to be African-American."

This isn't Carson's first go-round with the tea party. In March 2010, he accused protesters of yelling racial slurs at him and Rep. John Lewis, D-Ga., outside the Capitol during the heated health care overhaul debate.