Whether it's the rumors of buried treasure or of ancient curses, Milford's Charles Island has long held an air of mystery and intrigue. In our July/August 1972 issue, a woman who spent decades of her life on the island reflects on the sometimes idyllic — and sometimes eerie — experiences she had there.
This article is being posted to the web in October 2021 as part of Connecticut Magazine's 50th anniversary celebration.
My Lifelong Love Affair With A Haunted Island
By Gladys Walker
There are few things, few places in this land that do not change. One can rarely go back to a favorite childhood haunt and find it as one remembered. But I have been lucky. Charles Island stands seemingly untouched by time.
On a warm summer day this little island glints like a tiny emerald baguette in Long Island Sound. My first sight of it was probably at age six months on a late day in June, the time when my family migrated to the Milford shore for our annual summer stay. It is among my earliest memories, and so are the stories of its curses, its buried treasure and the many mysteries that shroud it.
The island lies a half mile off the Connecticut coast and is connected to the mainland by a narrow sandbar that is uncovered only at low tide.
The sandbar was about a quarter of a mile from our "cottage," which was a great hulk of a three-story building that looked more like a small hotel, complete with hip roof, full porches that wrapped around the first and second stories, and little balconies tacked on to side windows that gave the effect of medieval turrets.
We shared it with an uncle and aunt and two cousins, and we three children roamed the shore every day. Yet by some unspoken law we never went more than a few steps onto the island’s sandbar. It was as if we always knew there was a reason to avoid it, but my first recollection of any concrete evidence was the warning from our next door neighbor, Mr. Harper, who owned a boat livery.
“You kids keep away from that island," he admonished us. “There’s a real curse on it. Put there by an Indian chief when his daughter was kidnapped and taken captive out there. The chief cursed that island and swore nothin’ would ever stand on it. And nothin' has!"
Even as we listened to Mr. Harper, I remembered stories of raging fires and ravaging storms that had leveled island buildings. And it was true. Nothing had ever remained there for long. A tobacco plantation had been erected and destroyed. So had a mansion, a large hotel, a fertilizer factory, a religious retreat and many minor projects. But the Indian chief’s curse was only the first of three we were to learn about.
A trip to the island was not very long, yet we never made it more than once a season. My cousins and I would be chaperoned by our parents, or several young uncles and aunts, all of us with sneakers slung over our shoulders. For the sandbar was smooth enough, but as you approached the island pebbles grew to stones and finally turned into jagged rocks, making footgear a necessity. On the hottest summer day there was always a stiff breeze blowing across the bar, and the gulls never sounded more forlorn than they did here.
We looked for familiar landmarks —the ancient marine boiler bubbling just offshore, an old bedspring half buried in the sand here, the lacy rusting carcass of an ill-fated barge there. As we approached the island I always experienced a strange feeling of expectancy. Over the years, strangers who knew nothing of its legends sometimes professed to feelings of foreboding.
In those days it was owned by a religious order and used as a lay retreat. We were not permitted beyond the high water mark, but this did not prevent us from walking around the perimeter below the mark, nor making short, illicit forays along the paths that led to the interior. A tolling church bell or the sight of an unexpected shrine would send us flying out of the woods into the daylight.
It was always wise to keep an eye on the tide, for though it uncovered the bar starting from the mainland, when it finally turned again, the waters surged around the island first, and then back down the bar sometimes faster than you could run, stranding many a dawdler. Yet over the years, even with restrictions, we managed to do a remarkable amount of reconnaissance.
There was always a running series of local stories of hard luck and tragedy resulting from island ventures. Tales of mysterious currents kept us from putting even a toe into the deceptively inviting crystal waters off the bar where boats had in fact overturned for no reason, surf casters had gone under and disappeared, and strong swimmers had perished every summer.
We were content to stay with the Indian curse as the answer to these unexplainable mishaps until we heard the second legend, carrying with it the spine-tingling name of Captain Kidd, who was said to have landed on Charles Island one dark night. He buried his fabled treasure and put a curse on any who dug there until his return.
Mr. Harper moved away, leaving his house vacant except for closets full of conch shells that gave the place a distinctive aroma. We used his second floor as a secret lookout, peering at the island through my uncle’s ancient telescope. When we finally got our hands on a pair of reasonably good field glasses, we took turns recording observations in a log.
Kidd was hanged soon after in England, before he was able to retrieve any of his wealth. But rumors of treasure on the island spread, and one moonlit night three colonists, the story went, armed with shovels, stealthily made their way to the southeast corner where at least some of Kidd’s 216 chests of rubies, pearls and silks were reputed to be hidden.
They dug swiftly, but when they hit the top of an iron box, they looked up to see the figure of a man without a head coming toward them. They dropped their tools and fled, but glanced back just in time to see the ghostly figure go up in smoke and blue flames. When they returned by daylight, the spot where they had been digging was perfectly smooth, and there was no sign of their shovels.
Historical records later substantiated the fact that Kidd put into Milford harbor several times, once just before he was captured, and that he flirted with, even “kyssed,” one Patience Tuttle. And it would have been simple for him to slip into the far side of Charles Island, hide his booty and be off without touching the mainland, and no one the wiser.
But we didn’t need documentation at that point to persuade us that treasure was there. From that day on we adopted the island as our own. It served for years as one of our main amusements, and we spent hours of those deliciously free and endless summer days drawing maps and planning future excavations.
Our imaginations knew no bounds. Clammers, lovers, picnickers—none were spared. All were spy suspects. We guarded the island by day, and by night rested in our beds making up stories about bloodthirsty pirates and ghostly specters bringing down their curses.
As time passed we came upon the third curse one rainy afternoon in the library, a tale more fantastic than anything we could dream up. During the time of Cortez’ siege of Mexico City, the story said, Emperor Guatmozin hid his fabulous treasure in a cave, and put a curse on anyone who disturbed it.
Two-hundred years later it was discovered by some sailors and secretly loaded on their ship, The Salem Isle. Before long the captain, James Clark, found the treasure aboard and had to be made a partner. But within two weeks, four out of the five sailors who moved the rich cargo died mysteriously. The last survivor, Caleb Sayles, a lad from Milford, gave his share of the fortune to Captain Clark in an effort to shake the curse.
But Clark followed Sayles home, purchased the Post Road Inn for him, and persuaded him to hide the treasure there. However, a drunken customer discovered it and forced them to move it quickly. Old records showed a deed to Charles Island made out to Caleb Sayles and James Clark, and the island, it seemed, played host to a second treasure.
Two years later a small sloop put into the island for drinking water but its captain unearthed a brilliant necklace instead. He assumed the island was bristling with pirates, and left to get reinforcements. Later he attacked what turned out to be a deserted island, but Guatmozin’s curse lived on, it seemed. For somewhere on that tiny plot of ground the commander and second officer vanished and were never seen again.
The men were terrified by what one of them described later as "a silent air of mystery” which sent them running to their longboats, never to return again, because they feared “the earth beneath their feet would open and swallow them up.”
Our cottage was washed away in the disasterous 1938 hurricane. Shortly after my father found another one, much smaller and more modern to be sure, but without the primitive charm and hiding places of the old one. Gone were our turrets, gone were our log and field glasses, but the island had weathered the storm. Our new cottage was located on the opposite side of the sandbar, but about the same distance from the island as the old one.
The religious retreat had come on hard times and been abandoned, and the buildings were falling into disrepair. By now we were approaching our teens and were allowed to make unescorted trips. On our first expedition to the island we found the paths completely overgrown, but eventually we made it into a large clearing. What had been a church stood in the center, surrounded by tiny living quarters. We investigated crumbling outbuildings, and combed every inch of ground, discovering, among other things, the foundation of the old hotel. We charted everything.
Finally we made our way to the fabled southeast corner and spent a great deal of time there poking into and under everything. We sat on the shore’s edge for as much time as the insistent tide would allow, each of us lost in his own private world—part child, part adult.
Charles Island wasn’t ours for long, however. World War II came, and the island changed hands under a heavy cloak of mystery, and was closed to the public. Town rumors began to fly, favoring theories that it had been bought by the government for a submarine base or a secret training ground for special forces.
Our spying-in-earnest days were over. But by night, in an unaccustomed sea of darkness created by a wartime blackout, we were not above speculating about any flicker of light on the horizon, and deciphering imaginary SOS’s and signals to phantom foreign agents stationed on the island.
After the war was over and two of us had been away at college for a year, the island was again opened to the public. We decided to celebrate its liberation with a hot dog roast there. The evening was still and the water was like a tabletop when our party of three couples started out in a small rowboat. We covered the distance quickly. But, by the time we were ready to return, something had happened. The water was placid enough near our picnic site, but on the opposite side, where we had to launch, huge waves were washing over the bar. The stars had disappeared and the sky was black. A strong wind had blown up all of a sudden and we pushed off quickly.
Though two of the boys strained at the oars we seemed to be going backwards. The water splashed over the bow as the boat slapped down into troughs and crested on giant waves. In the inky blackness I lost all sense of direction in a small area I knew so well. After what seemed to be hours, by some miracle we came to shore a mile beyond our house, driven there by the current, and the boys pulled the boat back home by the anchor line.
We were greeted by a blinding searchlight. Behind it stood coast guardsmen, the police and my distraught parents. Until the moment I saw my father’s face, I had not had a moment of personal fear, and for all our years of fantasizing, no thoughts of curses or past tragedies had crossed my mind.
Soon after, the island’s mystery buyer was revealed as a public utility company which had purchased it as a possible site for a power plant in the distant future. A hew and cry went up but nothing happened.
In the ensuing years I took my own children out to the island. We inventoried the relics along the sandbar, many of them still there from my own childhood. As my children came upon the site of an old cross, or climbed the ruins of a chimney, I knew that another generation was being captivated by the island’s special magic.
Periodically a politician advocates getting the island back and clearing it for a marina, or worse, an amusement park. After the election furor, spring comes, the trees bud, a new crop of wild raspberries comes out, picnickers return, and visiting boatmen anchor in the inland's cove, oblivious to its mysterious past.
For centuries nothing has remained standing for very long on this island. Whether this has been brought about by curses, or by a fluke of nature stemming from some unique geographic location, what makes anyone think a new venture will succeed? The people of Milford, and I, continue to love this tiny piece of real estate. My money is on the wind and tide which continue to keep Charles Island ours.
RELATED: More ghost stories from the archives
From October 1977, “Stalking Witches and Boogeymen” visits several self-proclaimed mystics living in suburbia.
From Dudleytown to New Haven’s Midnight Mary, “Haunted Connecticut” (October 2015) outlines Connecticut’s most infamous supernatural legends.
From April 1972, “In Search of the Supernatural” profiles Connecticut’s O.G. ghostbusters, the late Ed and Lorraine Warren, still the most famous paranormal investigators in the world.
As recounted in “Bewitched” (October 1988), dozens of Connecticut residents were tried for witchcraft decades before the more famous trials in Salem, Mass.
We received a quite chilling reception when we first visited Cortlandt Hull and his Witch’s Dungeon museum of movie memorabilia way back in October 1984’s “House of Horror” (this month we pay a return visit to Hull at his new, expanded showplace in Plainville).
These and more articles from Connecticut Magazine's history can be found at connecticutmag.com/archives