I am married but I still have girlfriends. Is it an offence to engage in extra-marital affairs?

Such act is called adultery. Even though adultery is not an offence under the Marriage Act and the Criminal Code which applies in the Southern part of Nigeria, it is an offence in the North[1]. It is, however, an acceptable reason to file for divorce. A spouse who is aggrieved by the adultery or infidelity of the other spouse may file for divorce. For a court to grant dissolution of marriage on the basis of adultery, there must be convincing evidence which shows that truly adultery is committed by the other spouse, and that the spouse suing (i.e. the petitioner) finds it intolerable to live with the “adulterous” spouse. 

Whenever a divorce petition is presented on the basis of adultery, it is important to join a third party with whom a spouse commits adultery in that divorce case[2]. By joining the third party, the petitioner will be able to claim damages against the person and it will also afford the third party to state his or her own side of the story. The third party may, however, escape being ordered to pay damages, if the court is convinced that circumstances exist which may make the third party believe that the respondent is not married or if the respondent has created an impression of being unmarried. Mere suspicion that adultery is committed without any concrete evidence is not enough.

Extract from What the Law Says About Marriage and Divorce

[1]. It is an offence under the Penal Code and also a serious offence under the Sharia Criminal Codes of different Northern States that have passed such laws.
[2]. Ebe v. Ebe (2004) 3 NWLR Pt. 860, 215.