An acton-packed first half ended goalless, but a much more cagey second period saw Burgess rise highest to head home Carl Baker’s cross, and despite late pressure from the O’s, they slipped to defeat.
Cavasin made five changes from the 1-0 midweek Checkatrade Trophy defeat to Southend United with Alex Cisak, Paul McCallum, Jay Simpson, Yvan Erichot and Jens Janse returning to the starting XI.
It was a positive start form the O’s and inside a minute some good play down the right led to the first corner of the game. From the set piece the ball in proved to be a dangerous one to deal with and Paul McCallum’s header back into the danger zone was cleared by Pompey.
O’s continued in the ascendancy as McCallum and Simpson linked up well up top as the latter played in the former Pompey loanee who swivelled on the ball before seeing David Forde parry the effort away.
Cavasin and his side could breathe a huge sigh of relief on four minutes though when Chaplin produced an exquisite finish from a right-wing cross but the assistant on the far side ruled the effort out for offside.
An action-packed opening five minutes took a turn for the worse for Orient though when skipper Liam Kelly was stretchered off having come down in front of the dugouts and was stretchered off, and Sandro Semedo was his replacement.
That mean a midfield reshuffle once again, with Semedo moving to the left wing and Massey inside.
Pompey went close again as Linganzi was unmarked in the box from a Roberts corner but fortunately for the hosts his header was off target.
Huge penalty appeals were waved away by referee Christopher Sarginson on 11 minutes for the O’s as Yvan Erichot appeared to be hauled down in the box, however he decided to wave play on.
Cisak almost presented Pompey with a golden opportunity as a loose pass was picked up by Roberts, who in turn fed Chaplin but the Aussie made amends with a solid save.
The visitors, and most of the crowd, had thought they won a penalty on 23 minutes when Kyle Bennett went down in the box and the referee reached for his pocket and decided to book the Pompey man for diving.
It was Paul Cook’s side who had the chances to take the lead as the half drew to a close, with Bennett firing a shot just wide, and then Chaplin’s fierce effort from just outside the box was inches wide of Cisak’s goal.
Alan Dunne was brought on at the break in place of McCallum, with the O’s now going 4-2-3-1 with Dunne and Janse sitting in midfield.
And Dunne almost had an instant mpact as he picked the ball up 30 yards from goal and ran forward before trying to dink his former Millwall team-mate David Forde, however, the keeper was on hand to just claw it out of the air.
It was the visitors who would take the lead though and it came on 57 minutes when Christian Burgess headed home Carl Baker’s ball in.
Chances were few and far between as the half went past the halfway stage and the only real effort saw Roberts’ shot take a deflection on its way behind for a corner, while the O’s struggled to create opportunities.
Pompey came close to a second when Hunt headed a cross back on to his own post with Cisak scrambling.
Up the other end the O’s pushed forward in the last 10 minutes in the search of an equaliser. Callum Kennedy’s corner found the head of Tom Parkes but his bullet-like effort was blocked before both Parkes and Semedo saw their follow-up shots also kept out by the Pompey defence.
Try as they may the O's were unable to force a late equaliser, and new boss Cavasin will have a full week on the training ground with his squad ahead of next Saturday's home game with Luton Town.
Orient: Cisak, Hunt, Erichot, Parkes, Kennedy, Massey, Janse, Kelly (Semedo 9), Nnomo (Palmer 82), Simpson, McCallum (Dunne HT).
Subs: Sargeant, Bowery, Pollock, Benedicic.
Portsmouth, Forde, Stevens, Clarke, Burgess, Evans, Linganzi (Rose 67), Doyle, Baker, Bennett (Lalkovic 82), Roberts, Chaplin (Main 67).
Subs: O’Brien, Davies, Smith, Lalkovic, Naismith.
Booked: Bennett (29), Stevens (35)
Goals: Burgess (57)
Attendance: 6,078 (1,323 Portsmouth fans)
Referee: Christopher Sarginson.