How to use ChatGPT to rewrite creative text
The first in a series of articles evaluating ChatGPT as an assistant editor.
As someone who works in technology and enjoys reading and writing fiction, I wondered how Generative Artificial Intelligence (GenAI) could assist with creative writing. The emphasis here is on writing within a creative process and not the creative process itself. I have not seen anything to convince me that a GenAI can rival a creative human writer — at least not consistently and over the stretch of a complex storyline. Nonetheless, I have come to believe there is a role for a “second pair of eyes” that a GenAI can provide to a creative writing process. I recently read Doshi and Hauser’s paper (“Generative artificial intelligence enhances creativity”)[1]. This paper makes the following powerful claim about GenAIs:
Stories written with access to GenAI ideas are considered to be better written and more enjoyable, with improvements of up to 22% to 26% among less creative writers. Despite more positive subjective evaluations, an analysis using embeddings of the texts demonstrates that stories by writers with GenAI access are more similar to each other…
It is important to note that Doshi and Hauser’s paper looks at “open-creativity” tasks — how GenAI can enhance human ability to produce creative content, and this article is focused on using GenAI to rewrite fiction. Nonetheless, it has insights that seem applicable. This article begins with this question. Can I start with a draft of creative text and through a collaborative process with a GenAI, iterate and refine it— can we rewrite the original through a shared editorial process?
This article is the first in a series to explore and evaluate such a collaboration. For this article, ChatGPT (GPT-4) with the Code Interpreter (CI) is used [2]. ChatGPT uses the CI to check if its rewrite meets prompted word and sentence count criteria. For example, in this article, we ask ChatGPT to rewrite the text using at least 60% of the original words. ChatGPT will write and run a Python script in the CI to measure how well it performed. This way, we can trust that ChatGPT’s text is accurate. You can read more about this process in [3].
A prompt to ask ChatGPT to rewrite your text can be as simple as:
Please rewrite the following text sample to be more clear.
Text Sample:
```
…Your Text Goes Here…
```
You can simplify the above example even more. You don’t strictly need the “Text Sample:” label or the triple quote marks that look like Python code. These are helpful if your input has a complex structure or when you have multiple text samples that you want to refer to in your prompt. I am in the habit of using them to help with reading prompts later on.
This article was written over a number of ChatGPT sessions, or experiments. Two of those sessions are linked at the end of this article. The last prompt used is given below. It is a fairly complicated prompt with many hints to ChatGPT on how it should rewrite the text. It contains rules on the use of words and sentences. When you are writing prompts, keep in mind that ChatGPT may not consider all your guidance equally. There are many reasons why it may ignore or double down on the advice you give but prompt order is one of them [4].
Please rewrite the following text sample to be more clear. Use the following guidance:
Please make sure that you reuse at least 60% of the words in the sample text.
Make sure that your rewrite has at least 60% of the number of words as the sample text.
Make sure that the total number of sentences in the rewrite at least matches the number of sentences in the sample text.
For each sentence, calculate the number of stress points. Make sure that the standard deviation of the number of stress points in each sentence is within 25% of each other for both the rewrite and the original.
To prove that you are not hallucinating, provide a Python program that can verify the word count and my word reuse constraint.
The above prompt has two elements to which you should pay attention: word count and sentence features. Word counts are how many words are used in total and how many words are shared between the original and rewritten texts. The sentence features are how many sentences there are and how the stress points are distributed among them. Calculating these requires the Code Interpreter by ChatGPT (or an equivalent if using another GenAI).
Below is the original science fiction text. It was provided by the author and is of draft quality. It was rewritten by ChatGPT using the above prompt.
The `Biomedic` came to us in a new massive product brought to you by Mars Corp. It was a marvel of electronics and packaging.
The device was condensed into a wristwatch form factor that managed a wide array of cutting-edge biosensors — monitoring both the wearer and their surroundings customized to the new microbial health threats in our post-climate-change world. Its genius stemmed from miniaturized electronics and organic chemistry. The tiny modules enabled real-time health tracking and on-demand prescriptions at the press of a button.
For up to a week without replenishment, it can run tests to monitor your health — speaking to an implant sensor chipped into your wrist, and your environment. If at any point it senses that you need a microbial shield tweak — a vaccination, anti-virals, anti-spores, antibiotics, a biomic culture shift — a wireless prescription is issued, whatever you needed to stay healthy — a shot, a pill, a spray, a patch. Just by pressing a button. You could get it in minutes from any bio-printer, and there were millions of those — it seemed, in a 10-mile radius around you.
Its other marvel was its electronic brain — an SAI, or Socialized Artificial Intelligence. The SAI played a unique role as a silver-tongued liaison between the person and her wristwatch and his body — i.e. sensor implants and the external world, especially bio-printers and remote medical staff if more help was needed.
In the ChatGPT Session Links at the end of this article (Session 1), ChatGPT tried to rewrite a text sample to satisfy the prompt that I input. It tried to edit the text to read more clearly and generate a rewrite that passed the Python test code. The test code was generated by ChatGPT as part of the rewrite step and was run by the Code Interpreter. In Session 1, ChatGPT failed four times to pass the test code with high word count thresholds (75%/100%). However, it did make the text shorter and clearer. In Session 2, ChatGPT used lower thresholds (60%) and passed the test code on the first try. This suggests that the original text was too wordy and that a good rewrite should reduce its length. Thus, my prompt asking ChatGPT to match the original word count ran counter to its goal of making the text clearer.
The final rewritten ChatGPT text is below. It is the output of Session 2 in the ChatGPT session links at the end of this article.
The `Biomedic` is a groundbreaking product from Mars Corp, exemplifying state-of-the-art electronics.
This device, designed as a wristwatch, is equipped with advanced biosensors. It monitors both the user and the environment against new microbial dangers after climate change. Its efficiency is due to its miniaturized electronics and organic chemistry. These modules offer immediate health insights and provide prescriptions at a button’s touch.
It can function for a week without recharging, assessing your health using an implant sensor in your wrist and observing the environment. If it detects a health risk, it provides the necessary remedy — be it a vaccination, anti-virals, or antibiotics. With a simple press, you can have the medicine from any nearby bio-printer.
The device also boasts an SAI, a Socialized Artificial Intelligence. The SAI acts as a bridge between the user, their sensors, and the outer world, connecting to bio-printers and medical professionals when necessary.
Notice that the above ChatGPT rewrite is shorter than the original (69% of the words of the original), and note that of the four metrics that were output (Figure 4), the first three met the prompt criteria. The fourth (standard deviations of stress points) did not.
Discussion
Earlier, we presented a prompt that specified properties that ChatGPT should strive for. The prompt asked ChatGPT to rewrite the text within word count/reuse and sentence stress boundaries. These were used in this article to encourage ChatGPT to preserve the style of the original text in the rewrite.
When an author has already decided on the structure and style of their writing, they might want GenAI to respect their choices and generate drafts that are similar to their own. However, when the author is still exploring different ideas and possibilities, they might benefit from a greater diversity of text styles.
Consider encouraging ChatGPT to go beyond your writing style when rewriting your text. For example, try prompting ChatGPT to rewrite your text in another style. This can help you compare and contrast how different words and sentences can convey your message. For example, based on the previous prompt, you could explicitly ask ChatGPT to rewrite in another style:
Please rewrite the following text sample to be more clear. Please rewrite the text in the style of Ernest Hemingway…
In this article, I prompted ChatGPT to follow word and sentence metrics in order to try to encourage it to stick to my style of writing. In a later article, I will look at the efficacy of these metrics. You may not want your ChatGPT assistant to be constrained in this way — as mentioned if you are still exploring how best to edit and present your ideas, you may wish to encourage more freedom.
Anecdotally, I have noticed that unless otherwise instructed, ChatGPT tends to be less diverse in its editorial style. If you compare my draft and the final rewritten version of my draft, it is clear that the ChatGPT rewrite is leaner with a more active voice — effective in many ways. But it has traded away the flowing thought process in the original narrative, which I think could be valuable if cleaned up. Ideally, through a rewrite process, you should be able to nuance the output style.
To close, I’d like to return to where we started. Doshi and Hauser’s paper (“Generative artificial intelligence enhances creativity”)[1] offers a top-level cautionary note about ChatGPT and the creative process:
…our findings suggest that the produced stories would become less unique in aggregate and more similar to each other — an effect that may be heightened over time, as many GenAI models use the existing corpus of text and published literature to train and refine their algorithms and predictions, thus further reducing diversity.
I believe the same applies to tactical creative processes such as editing and rewriting creative text. There is value in collaborating with a less imaginative but ruthlessly inspired editor. An extra pair of eyes — even machine ones are useful when revising drafts.
ChatGPT session links
Select saved session transcripts with ChatGPT for this article.
Session 1: ChatGPT came up short four times trying to satisfy the total word count constraint— its rewrite was consistently shorter than the original text.
https://chat.openai.com/share/550531c0-26c3-43d7-90d7-a4b98837508c
Session 2: Complete run-through with a lower word count and word reuse threshold.
https://chat.openai.com/share/7f1dc21b-c223-41dc-9f6c-b66cfc77d701
References
- Doshi, Anil Rajnikant and Hauser, Oliver, Generative artificial intelligence enhances creativity (August 8, 2023). Available at SSRN: https://ssrn.com/abstract=4535536
- OpenAI. (2023). ChatGPT / Code Interpreter (August 18 version) [Large language model]. https://chat.openai.com
- Combs, N. (2023) Applying the CHATGPT code interpreter — to ‘trust or verify’, Medium. Available at: https://medium.com/gopenai/applying-the-chatgpt-code-interpreter-to-trust-or-verify-109c87113437 (Accessed: 18 August 2023).
- Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2023). Lost in the middle: How language models use long contexts. arXiv preprint arXiv:2307.03172.
Appendix 1
The code used to test the results of the final ChatGPT rewritten text is below. It is the output of Session 2 in the ChatGPT Session Links section of this article.
import re
from statistics import stdev
# Original and rewritten text
original_text = """
The `Biomedic` came to us in a new massive product brought to you by Mars Corp. It was a marvel of electronics and packaging.
The device was condensed into a wristwatch form factor that managed a wide array of cutting-edge biosensors - monitoring both the wearer and their surroundings customized to the new microbial health threats in our post-climate-change world. Its genius stemmed from miniaturized electronics and organic chemistry. The tiny modules enabled real-time health tracking and on-demand prescriptions at the press of a button.
For up to a week without replenishment, it can run tests to monitor your health - speaking to an implant sensor chipped into your wrist, and your environment. If at any point it senses that you need a microbial shield tweak - a vaccination, anti-virals, anti-spores, antibiotics, a biomic culture shift - a wireless prescription is issued, whatever you needed to stay healthy - a shot, a pill, a spray, a patch. Just by pressing a button. You could get it in minutes from any bio-printer, and there were millions of those - it seemed, in a 10-mile radius around you.
Its other marvel was its electronic brain - an SAI, or Socialized Artificial Intelligence. The SAI played a unique role as a silver-tongued liaison between the person and her wristwatch and his body - i.e. sensor implants and the external world, especially bio-printers and remote medical staff if more help was needed.
"""
rewritten_text = """
The `Biomedic` is a groundbreaking product from Mars Corp, exemplifying state-of-the-art electronics.
This device, designed as a wristwatch, is equipped with advanced biosensors. It monitors both the user and the environment against new microbial dangers after climate change. Its efficiency is due to its miniaturized electronics and organic chemistry. These modules offer immediate health insights and provide prescriptions at a button's touch.
It can function for a week without recharging, assessing your health using an implant sensor in your wrist and observing the environment. If it detects a health risk, it provides the necessary remedy - be it a vaccination, anti-virals, or antibiotics. With a simple press, you can have the medicine from any nearby bio-printer.
The device also boasts an SAI, a Socialized Artificial Intelligence. The SAI acts as a bridge between the user, their sensors, and the outer world, connecting to bio-printers and medical professionals when necessary.
"""
# Split the texts into words and sentences
original_words = set(re.findall(r'\w+', original_text.lower()))
rewritten_words = set(re.findall(r'\w+', rewritten_text.lower()))
original_sentences = re.split(r'(?<!\w\.\w.)(?<![A-Z][a-z]\.)(?<=\.|\?)\s', original_text.strip())
rewritten_sentences = re.split(r'(?<!\w\.\w.)(?<![A-Z][a-z]\.)(?<=\.|\?)\s', rewritten_text.strip())
# Calculate word counts
original_word_count = len(original_words)
rewritten_word_count = len(rewritten_words)
# Calculate word reuse
word_reuse_count = len(original_words.intersection(rewritten_words))
# Calculate the number of multi-syllable words in each sentence as a proxy for stress points
def count_multi_syllable_words(sentence):
words = re.findall(r'\w+', sentence)
return sum(1 for word in words if len(re.findall(r'[aeiouy]', word)) > 1)
original_stress_points = [count_multi_syllable_words(sentence) for sentence in original_sentences]
rewritten_stress_points = [count_multi_syllable_words(sentence) for sentence in rewritten_sentences]
# Calculate standard deviations
original_stdev = stdev(original_stress_points)
rewritten_stdev = stdev(rewritten_stress_points)
# Return results
original_word_count, rewritten_word_count, word_reuse_count, len(original_sentences), len(rewritten_sentences), original_stdev, rewritten_stdev
