Ghost Walks Past

I was working in an 19th Century building in Portsmouth UK. The building was owned by the council and was once a barracks. It was like a castle and the upstairs had been turned into offices with sprung floors and partition walls. At about 5.45pm on a November late afternoon, dark outside, I was alone in the office I shared with two others. I was finishing at the PC and I heard one of the caretakers walk past my door and call out that he was locking up in approx 20 minutes. I shouted out 'OK'. This was a usual routine and completely normal. He walked away down the stairs and I heard the doors shut behind him etc. I took about ten more minutes to finish and then heard distinct footsteps walking back the other way. I called out "I'm just switching off". I did this, picked up my things and left. This building is on three floors and I walked down the spiral stairs and into the foyer. The caretaker was sitting behind the main desk reading a paper. I was surprised to see him as I expected him to be over the other side of the building due to hearing steps just passing my door heading towards the other parts of the site. I said 'sorry to be late' but I asked how he had 'got down so quick?' He said he'd been sitting at the desk for about 10 minutes. I asked him if he was sure and he repeated that he's been there since walking past earlier. I said there must be someone else upstairs because I heard someone walk past. We agreed the details of when and where he had walked past towards the foyer and not that he'd returned. He said that others had heard similar steps in the building at this time of day. No one else was on duty and we were the only people there. I definitely heard footsteps and felt the floorboards move as the person walked past. The site had in-room cctv and a succession of locked doors which he had locked as part of the closing up process. The only way out at that time was through the foyer area. We both walked the site and found nothing. I am convinced that something was there...