The two output pins of the NJM2113 audio amp chip (U1) are each biased at about 6 volts DC. That's why they can't be allowed to be grounded - it shorts out the bias voltage and zaps the output stage.
So you are correct - if you are using polarized capacitor(s) for DC blocking, the positive lead(s) should be connected towards the chip's pin(s)
You might get away with just one capacitor - in series with the ring terminal. That's the terminal that's likely to get grounded if using a powered speaker, or a speaker with a mono plug that has a ground path. But you have to be sure that the tip terminal of your external device doesn't have a DC path to ground somehow. Many powered speakers or audio filters already have a blocking capacitor in series with their input. Check their schematic or physically inspect the device if possible. If you're not sure, put caps in both leads.
Value depends on what is to be connected. Many powered speakers or audio filters have a fairly high input impedance, so a smaller capacitor is sufficient.
For example - suppose the input impedance is 100K. At 300 Hz, even as small as a 0.1 uF cap has a reactance of only about 5K. So it would not introduce very much voltage drop. But if the input impedance were like that of a normal speaker or headphone (a few tens of ohms), very little signal would get through.
This is why putting the capacitor in the external cable can be a convenience. You won't interfere with the normal use of your headphones, and only a smaller value capacitor(s) will probably be needed.
If the capacitor(s) get installed between the PC board and the headphone jack, and if you still intend to use headphones on occasion, the the capacitor(s) will need to be much larger - probably at least 10 uF, but need not be larger than 50 uF or so.