Michael Ademiluyi had handed the hosts a fifth minute lead with a low strike, but after trailing at the break Orient responded through Coulson who volleyed home on the 58th minute.
The game was then turned on its head 11 minutes from time, as Harrold came off the bench to score with his first touch and head Orient into the next round.
On the back of last weekend’s 4-1 victory at home to Sutton United in the league, Justin Edinburgh made two changes to the starting XI as David Mooney came in for Matt Harrold and Dan Happe dropped to the bench in place of Josh Coulson who returned from a groin injury to make his first appearance since September 23.
Jamie Sendles-White also returned to the bench, but the O’s remained without Jake Caprice who served the final game of a three-match suspension.
With a record crowd in attendance at Coles Park, Orient got the FA Trophy tie underway but following a cagey opening few minutes fell behind on the fifth minute when Michael Ademiluyi broke free and drilled a low shot beyond the reach of Dean Brill.
Macauley Bonne then went in search of an immediate equaliser when a cross dropped into his path inside the area, but despite connecting well with the strike Rakim Richards bravely blocked the effort.
Former O Harrison Georgiou then touched wide an Anthony McDonald cross on the 12th minute, before Bonne squandered two chances in as many minutes when he first fired over from a David Mooney knockdown and then glanced a header over the bar.
A brilliant cross from James Dayton then presented Mooney with a great chance to level on the 20th minute, but the Irishman who only direct his header wide of the post.
Both sides continued to exchange chances and McDonald soon followed up with an effort which flew wide, and with just under 15 minutes of the half remaining Orient had a wonderful chance to equalise when Mooney slipped in Dayton but the midfielder’s strike was parried away by Ashley Harris.
Brill had been a spectator for much of the half but with 34 minutes on the clock had to be at the top of his game to get down low to his left and superbly tip a McDonald effort wide after the winger had done well inside the box.
Orient penalty appeals were then waved away soon after as Mooney went down under the challenge of Quba Gordan and the final chance of the half fell to Bonne who saw his header well held by Harris from a Dayton free-kick as the O’s went into the break a goal down.
Following the interval the O’s returned to the field unchanged, and within three minutes of the restart a Joe Widdowson cross was glanced over by Bonne as they went in search of an equaliser.
Craig Clay then rifled an effort inches over the bar on the 52nd minute, and as the O’s upped the pressure Bonne directed a header straight at Harris after Alex Lawless had regained possession well inside the hosts half.
Orient’s pressure was then rewarded on the 58th minute as they finally found a way beyond Harris to equalise.
Captain Jobi McAnuff whipped in a corner towards the far post which found the returning Josh Coulson, and the defender made no mistake by rifling home his first goal for the Club in off the bar before sprinting straight over to the O’s medical staff to celebrate.
The goal proved to be Coulson’s final involvement in the tie, as a minute later he was replaced by Dan Happe in a like-for-like change.
Having found an equaliser, the O’s then turned their focus on grabbing a second, and Bonne headed wide before Clay saw a low effort tipped wide by Harris.
Clay then looked to have given Orient the lead on the 68th minute when he curled a brilliant strike beyond Harris who was left rooted to the spot, but the ball crashed back off the post to deny the midfielder.
Haringey then announced a record attendance of 1,133 at Coles Park, almost trebling their previous best of 401.
With just over 10 minutes remaining Head Coach Justin Edinburgh introduced Matt Harrold in place of Lawless, and within seconds of his introduction the forward restored the O’s lead as he got on the end of a McAnuff corner to head home with his first touch.
Harrold’s impact had turned the game in Orient’s favour, and with three minutes remaining the game could have been sealed when McAnuff let fly from outside the area but his low strike whistled wide of the post.
As the clock ticked down Haringey reverted to a more direct approach, and a long throw from Lauren Luke saw Jordan Edwards connect with a header which needed to be tipped over by Brill for a corner.
Seconds later another long throw from captain David Olufemi presented Gordon with a chance, but his header was straight at Brill who gratefully held the attempt.
Five minutes stoppage time was then shown by the fourth official, but the O’s were able to see of late pressure from the hosts to book their place in the Second Round.
HARINGEY BOROUGH: Harris, Olufemi (C), Mitchell (Edwards 81), Gordon, O’Donoghue, Ademiluyi, Hervel, Richards (Luke HT), McDonald, Georgiou (Heather 73), Benjamin
SUBS: Byfield, Coley
GOALS: Ademiluyi (5)
BOOKED: Georgiou (57)
ORIENT: Brill, Judd, N’Gala, Coulson (Happe 59), Widdowson, McAnuff (C), Lawless (Harrold 79), Clay, Dayton (Sendles-White 90+3), Mooney, Bonne
SUBS: Grainger, Koroma
GOALS: Coulson (58), Harrold (79)
BOOKED: Judd (65), Mooney (83), Widdowson
REFEREE: Gary Parsons