Clemsons recruiting would suffer in the Big 12. I can't imagine a lot of southeastern football recruits wanting to go play football in a midwestern league. It is already having a negative impact on recruiting. Link.

The schools in your division are going to matter, after all, these are the schools you would play every year.

Big 12 East

Florida St.
Iowa St.
Kansas
Kansas St.
West Virginia

OR

SEC East

Florida
Georgia
Kentucky
NC State
Tennessee
Vanderbilt
Virginia Tech

"Sorry Dabo, I'm signing with Spurrier."

I can't speak for NC State, but you can pretty much book it that Virginia Tech would go to the SEC if Clemson and FSU left for the Big 12. The consensus among most VT insiders is Missouri's spot in the SEC was Virginia Techs for the taking. That could open up the flood gates of teams leaving causing the destruction of the ACC.

A decade from now, if the long horn network is a bust, Texas, Texas Tech, Oklahoma, Oklahoma St., could decide the time is right to join the PAC 12. If that happens, a move by Clemson and FSU to the Big 12 could be seen as one of the biggest blunders in sports.