First of all I suggest you to try with different sizes of the icon which you want, without no paddings until you achieve your goal. If that not suits you, you can add a custom view in actionbar and set OnClickListener
on a button and achieve your goal. Here is a way how to do this :
View customNav = LayoutInflater.from(this).inflate(R.layout.my_button, null); // layout which contains your button.
Button mButton = (Button) customNav.findViewById(R.id.date_num);
mButton.setOnClickListener(this);
getSupportActionBar().setCustomView(customNav);
getSupportActionBar().setDisplayShowCustomEnabled(true);
That should do the trick. You can use Button or ImageButton to achieve this.