GH-48961: [Docs][Python] Doctest fails on pandas 3.0

[tadeja]

Rationale for this change

See issue #48961
Pandas 3.0.0 string storage type changes https://github.com/pandas-dev/pandas/pull/62118/changes
and https://pandas.pydata.org/docs/whatsnew/v3.0.0.html#dedicated-string-data-type-by-default

What changes are included in this PR?

Updating several doctest examples from string to large_string.

Are these changes tested?

Yes, locally.

Are there any user-facing changes?

No.

Closes #48961

• GitHub Issue: [Docs][Python] Doctest fails on pandas 3.0 #48961

[AlenkaF]

Thank you @tadeja for looking into this!

One question regarding the bump of the Python version in Sphinx&Numpydoc job. I think it would be good if the examples worked for users with new or old pandas version. What if we use ... (ELLIPSIS) instead of changing the string type? Or even better, we could not use pandas where possible and instead create a pyarrow table directly, like so:

arrow/python/pyarrow/table.pxi

Lines 1812 to 1814 in 95a3ed4

	>>> table = pa.Table.from_arrays([[2, 4, 5, 100],
	... ["Flamingo", "Horse", "Brittle stars", "Centipede"]],
	... names=['n_legs', 'animals'])

[rok]

Agreed that it doesn't make sense for us to "test Pandas logic" especially in our docs. Agreed with @AlenkaF to instantiate the table in pyarrow. Using ellipsis in this case would hide the type and potentially increase user confusion :).

[AlenkaF]

Note that some examples are demonstrating conversion from pandas to pyarrow so in that case we might remove the string column and only keep the integer ones?

[rok]

This looks good to me now. I think (hope) removing pandas from examples that don't require streamlines things for readers.

[rok]

Interesting, I wasn't aware this works.

[AlenkaF]

Nice!

[rok]

I think this is a good change, just pointing out there is some interesting behavior here.

[rok]

@github-actions crossbow submit preview-docs

[github-actions]

Revision: 186c0a9

Submitted crossbow builds: ursacomputing/crossbow @ actions-ca47b1b8be

Task	Status
preview-docs

[tadeja]

@AlenkaF this is ready for final review.

• Generated doc pages: pyarrow.Table page and pyarrow.RecordBatch

• Both Sphinx jobs ran and completed doctests with success;
  AMD64 Conda Python 3.12 Sphinx Documentation
  pandas 3.0.0 pypi_0 pypi
  ================== 385 passed, 2 skipped, 1 warning in 6.24s ===================
  and
  AMD64 Conda Python 3.10 Sphinx & Numpydoc
  pandas 2.3.3 pypi_0 pypi
  ======================== 385 passed, 2 skipped in 5.63s ========================

• The two trivial cases where pandas 2.3.3 output expects None but pandas 3.0.0 expects NaN
1 4 None 2022.0
1 4 NaN 2022.0
  get best resolved by populating pa.array with a string instead: first case and second case.

• Note that I additionally removed pandas and replaced with pyarrow table for these three examples: def itercolumns, def remove_column and def join (although these are currently not causing failures as there isn't string vs. large_string in their output).
  But there are more unnecessary pandas examples remaining that could be simplified in the future (num_columns, num_rows etc).