The Blues have spent the past six days embroiled in the controversy over inappropriate comments referee Clattenburg is alleged to have made to Blues midfielder John Obi Mikel during Sunday's 3-2 defeat to Manchester United.
Victor Moses put Chelsea ahead before Pablo Hernandez's late equaliser and Di Matteo said: "I wouldn't read anything into what has happened in the last week and the result today. It is very difficult to win away from home in the Premier League."
He added: "This is our second away draw of the season - we have won the rest of the games. It is disappointing not to have hung on to our lead but even when you are not brilliant, you need to get a result like that.
"It's a long season and it is going to stay very tight at the top. There are a few teams fighting for that. We are in touch and we are going to show how strong we are in the future."
Swansea merited a point after having the better of the opening half and playing with a swagger reminiscent of their best form of last season, although they struggled to break down a Chelsea rearguard well-marshalled by Branislav Ivanovic and Gary Cahill.
Hernandez's low strike capped a good week for the Swans, who were narrowly defeated by Manchester City before dumping Liverpool out of the League Cup at Anfield.
He said: "I think it was a deserved point. It was, in many ways, a copy of the game we played last week against Manchester City. That time we deserved a point but didn't get anything, we did this time.
"They had a few goal chances, we had a couple, two or three half chances. There were not that many chances. A point for each was a fair result. Their goal was a bit lucky but we came back, got some fresh people on the pitch and really put them under pressure. I think we deserved it. It was a great goal by Pablo.
"It has been an important week for the club. we have had three different results the last week but three very good performances. It's nice to see that we can play like that against the big teams in our way of playing. That's important for us."
Source: PA
Source: PA