The eerie grave of 200,000 monks

It was dusk when we entered the cemetery. A stone path, faintly lit with lanterns, snaked under the towering hemlock and umbrella pine trees. We cautiously walked down it and plunged into the embrace of a sacred 1,200-year-old forest. Flickers of light bounced off ancient graves, shadows moved through the thick incense-scented woods and faces carved into stone eerily peered at us from the graveyard’s blackest corners. It felt like we were being watched. And perhaps we were. After all, this wasn’t just any cemetery. It was the misty and mossy Okunoin Cemetery, Japan's biggest at 2km-long and home to more than 200,000 graves of Buddhist monks who are said to be waiting for the resurrection of the Future Buddha. It dates back to at least 816AD, and every inch of it is sacred. But right now, it only felt creepy.

Okunoin Cemetery is in Koya-san, an ancient village located in Japan’s mountainous Wakayama Prefecture. It’s just one of many sacred places in the Kii Mountain Range collectively inscribed on Unesco’s World Heritage list; also included are the region’s Shinto sites and the ancient pilgrimage paths that connect them to more recent places of Buddhist worship, introduced to Japan 1,500 years ago by monks from modern-day Pakistan and Afghanistan.

Koya-san is the epicentre of Shingon Buddhism, which is one of the few surviving Tantric lineages of Buddhism in East Asia and is known for its mysterious mikkyō: secret monastic teachings handed down orally through an initiatic chain. The sect was introduced to Japan in 805AD by poet and Buddhist master Kūkai, also known as Kōbō-Daishi, who studied with monks in Xi’an, China, during the Tang Dynasty and remains one of Japan’s most significant religious figures. In a mausoleum at the end of the cemetery, he’s offered ritualistic meals twice a day where he remains in eternal meditation concentrating on liberating all beings. Or as we say in the west: dead.

My husband Ralph had arranged our visit to Koya-san as part of his two month sabbatical in Japan. Like most travellers to this remote mountaintop, he was interested in experiencing a spiritual retreat at one of Koya-san’s 54 shukubo: inns where guests are encouraged to meditate, commune and eat vegetarian cuisine with the monks or nuns. A stay here promised a tranquil antidote to the heavy drinking, eating, shopping and karaoke-ing we’d been doing in Tokyo.

Getting to Koya-san prepared us for the quiet to come. As with most places in Japan, bullet train is the most efficient way to get there. The five-hour trip from Tokyo transferred onto to the scenic Nankai Line at Osaka’s Namba Station, from where it left the frenetic urban flatlands of the Kinai Plains and climbed into soft green hills peppered with cherry blossom trees, Japanese ryokan(old-fashioned guests houses) and half-timber 9th-century buildings. The last leg of the journey was a scenic five-minute funicular ride from Gokuraku-bashi Station that shot straight up an 800m-high mountain, leaving the modern world and all its trappings below.

“It feels like we’re going to heaven,” I said, as we ascended into delicate mountain light.

It’s said that Koya-san’s eight different peaks resemble a lotus flower. I don’t know if that’s true, but after arriving, it was easy to see why this fog-choked summit was chosen as a spiritual place by early monastic settlers. An impenetrable stillness filled the cold spring air.

The cable car deposited us at the bus depot, where we jumped on a five-minute bus-ride to village’s east end. The bus dropped us outside two neighbouring but very different shukubo. One had a trickling mossy-banked stream worming through a gorgeous garden with elegantly gnarled cedar and sculpted topiary, while the other had a giant Korean tour bus parked in front of flashing electric screens bedazzled with rainbow Buddha and snarling Guru Rimpoches. Ours, Sekishou-in, was the one with the tour bus.

Buddhists speak frequently about the nobility of poverty, but the cost of this spiritual retreat, booked through the Koya-san Shukubo Association, was not for the impoverished. Our room, dinner, breakfast and meditation cost 32,000 yen per night, and a confirmation email reminded us it was cash only.

Sekishou-in was disappointing. The 60-room inn, one of town’s largest, was popular with groups, and visitors thundered through the creaky wooden hallways to the temple during meditation and meal times. Dinner was indeed prepared by the monks, but we didn't eat with them, rather in assigned halls with other foreigners.

Sekishou-in’s saving grace was our room, which was as old fashioned and elegant as a five-star ryokan, complete with tatami mat floors, sliding fusuma doors and floppy futon mattresses. A small deck with two small chairs overlooked a shared courtyard filled with maple and cypress trees and reflective ponds, making it near impossible not to meditate.

The next morning we set off to explore a few of Koya-san’s 117 temples and monasteries. We started with a five-minute climb to the Kiyotaka Shinto Shrine, perched on a nearby wooded hill and marked with a row of cinnabar-painted torii (thegateways that mark Shinto areas), before walking 15 minutes onwards to the main Buddhist temples in the Danjo Garan Complex, an active monk training centre and the town’s highlight.

The complex includes the 9th-century Konpon Daito, a vermillion-coloured double-decker pagoda decorated with 16 bodhisattvas, including the eight patriarchs of Shingon Buddhism. Unlike most of Japan’s understated Zen Buddhist temples, the iconography here featured wild depictions of deities and brightly-coloured mandalas more typical of Tibet, Bhutan and India. We also peeked inside Kondo – the Golden Hall where Guru Bhaishajya, the Medicine Buddha, is enshrined – and heard the deep “thung” of the copper Daito Bell, Japan’s fourth largest bell. After we got our fill of the complex, we walked across town to the 17th-century Edo-era Tokugawa Family Mausoleum, built for the famed father and son shoguns. Koya-san may be a major tourist draw, but its density of sacred sites helped us avoid the crowds we experienced in the ancient capitals of Kyoto and Kamakura.

Later that afternoon, we found ourselves inexplicably drawn back to Okunoin Cemetery to explore the paths we hadn’t the previous night. By day, Okunoin was everything the darkened cemetery was not. Bronze shards of light pierced through the canopy, illuminating the trees and moss and multi-tiered stupas lording over the gravestones. It felt as if it all belonged here for eternity.

The faces we had seen the night before were stone statues of the bodhisattva Jizo Bosatsu, who came in many sizes: tall and lean, short and stout, even miniature and hidden like Easter eggs in the crooks of tree-trunks. Jizo is usually depicted smiling, and sometimes, pink rouge is applied to his cheeks, making him as jovial as an anime character. Visitors who have lost children place red bibs on the Jizo statues so that he’ll watch over them.

There were hundreds, maybe thousands of fresh aprons flapping in the gentle wind around us. Even for a hardened traveller like me, it was hard not choke up over the quiet and heartbreaking dignity of it all. This was a landscape that had absorbed the pain and suffering of others. It was achingly beautiful.

At the end of the cemetery path was Torondo, the Lantern Hall, used as a gateway to Kūkai’s mausoleum and where 10,000 donated lanterns hang. Two of these lanterns have been burning continuously since 1088AD, one from a former emperor and the other from a peasant woman who sold her hair for a lantern to pray for her deceased parents. The hall was about to close just before we arrived, so we quickly slipped inside to admire its ornate woodwork and hand-carved lanterns while the sun began to sink and darkness once again settled around us.

But this time, the cemetery wasn’t spooky, and the difference between light and dark felt meaningless. I thought of the Buddha’s last words before beginning his own eternal meditation, “Be a light unto yourself”, and with that we walked back to the shukubo, finally illuminated from within.

If you liked this story, sign up for the weekly bbc.com features newsletter, called “If You Only Read 6 Things This Week”. A handpicked selection of stories from BBC Future, Earth, Culture, Capital, Travel and Autos, delivered to your inbox every Friday.