In September 2011, I spent long hours sitting on the beach in Muscat, Oman, with my father-in-law. We’d venture out early and sit wordlessly side-by-side, watching the ocean turn from grey to crystal blue. When words did come, they would mostly be questions—daring glimpses into the future or awkward attempts to come to grips with my recent past.

“I think the best way to describe solitary confinement,” I told him, “is that it feels like a slow death. You’ve lost everything yet somehow miraculously you’re still alive … Then you realize that you can’t live that way for long without dying inside.”

Years later, it’s still difficult for me to understand and articulate what spending 410 days of solitary confinement in an Iranian prison did to my body, brain and personality—let alone the ripple effects this practice has on our society as a whole. When I came back to the U.S., I was shocked to find this draconian punishment is more prevalent in our prisons than in any other country—any country in history. We hold at least 80,000 prisoners in some form of isolated confinement on any given day—that’s 22-24 hours locked in a small cell for months, years, even decades. Our prison system uses solitary confinement as a routine practice instead of a last recourse—as it is assumed to be.

Convinced that solitary confinement is an ineffective, dangerous and costly practice antithetical to any pretense of rehabilitation, I’ve spent the last four years advocating against its use through journalism, advocacy, theater and a memoir I co-authored with my fellow hostages (A Sliver of Light: Three Americans Imprisoned in Iran). During that period, it’s been heartening to see this subject increasingly discussed by journalists, human rights advocates, psychology experts, activists and survivors. A handful of states have substantially reduced the number of prisoners in isolation, various forms of state legislation have been passed and a few months ago, Attorney General Eric Holder criticized the use of isolation among our prisons’ juvenile and mentally ill populations.

As hopeful as this may sound, I fear reforms we’ve seen have far too narrow a focus. Of course, solitary confinement should never be used against the mentally ill, against juveniles or against rape victims and LGBTQ prisoners “for their protection,” but by focusing legislation and activism on the most vulnerable populations in our prison system, the cruelty and ineffectiveness of this practice as a whole may be lost.

The U.S. holds at least 80,000 prisoners in some form of solitary confinement.

When I tell people about my experience in Iran, they are usually horrified. I was an innocent pawn, caught up in a geopolitical game, of course I didn’t deserve conditions that UN Special Rapporteur on Torture Juan Méndez and many social scientists consider psychological torture. Yet, when I shift the conversation towards the work I do now—working against the same or worse conditions in the U.S.—these thoughtful, sympathetic eyes often glaze over. The most common assumption is that the people who end up in “the hole” must be so violent and dangerous they can be dealt with in no other way.

Let me dispel that myth.

The majority of people subjected to prolonged isolation are not there for violent acts, but petty prison infractions like talking back to a guard, walking too slow or possessing contraband. Prison administrations are guilty of using solitary confinement as a way to disappear and punish prisoners who annoy, inconvenience or threaten their authority. Some prisoners did commit violent acts against guards or other prisoners to get there but our prisons are extremely violent places. Locking people with anger issues in small, confined spaces reduces their impulse control­, therefore, increasing violent tendencies. Isolating prisoners not only postpones the myriad problems incubated in our nation’s prisons, it cultivates them and then dumps them back into our streets.

We think using harsh punishment makes us safer. It doesn’t. Perhaps the questions should not be whether prison administrators “have a choice” other than using solitary confinement but whether we, as a society, can afford to allow prison administrators to make these decisions on our behalf. Isolating prisoners is not only dangerous and inhumane—it’s also expensive, in large part because of added staffing costs. One study estimates the average per-cell cost of housing an inmate in a supermax (isolation) prison is $75,000, as opposed to $25,000 in the general population.

As I began to tease out what had happened to me in isolation, I realized my recovery was going to be tied up with that of many others: my family, my fellow captives who were released a year after I was, and perhaps even the tens of thousands of people in my own country subjected to similar inhumanity. Anything short of a nationwide ban on the use of prolonged solitary confinement—defined by the U.N. as 15 days—will leave tens of thousands of people locked in these torture chambers of the mind. The use of solitary confinement must be condemned and abhorred wherever it is practiced.

Sarah Shourd is a contributing editor at Solitary Watch, a web-based advocacy and information project focused on solitary confinement.