Qwen3 Early Exploration
I’ve been experimenting with large language models for a while now, primarily using the 1.5B parameter Qwen2.5 model through python and Huggingface’s Transformers library.
Even with my modest setup, a NVIDIA 3060 GPU, I could comfortably generate impressive responses.
#!/usr/bin/env python
import sys
from transformers import pipeline
def main():
"""
"""
max_tokens = int(input("max_token: "))
pipe = pipeline(
"text-generation",
model="Qwen/Qwen2.5-1.5B-Instruct",
device="cuda"
)
messages = [
{"role": "user", "content": sys.argv[1]}
]
res = pipe(messages, max_new_tokens=max_tokens)
output = res[0]['generated_text'][-1]['content']
print(output)
# entry
if __name__ == '__main__':
main()
Let’s first look at some of the results I would have gotten with Qwen2.5.
Throughout this post I’ll use the following prompt:
write an engaging introduction to a fantasy world for a dungeons and dragons campaign, this is a wilderness setting in a high fantasy world, describe it in detail
Notice I also adjust max_token as in the past I’ve ran into GPU out of memory failures depending on the generation size.
$ ./qwen_2.5.py $PROMPT max_token: 500
In the vast and unforgiving expanse of Ardenia, where the air whispers secrets from forgotten lands and ancient magic dances across the horizon, lies a realm that has been both revered and feared by generations. The land stretches as far as the eye can see, its terrain shifting like shifting sands underfoot - towering mountains, jagged cliffs, lush forests, and winding rivers weaved through a tapestry of rolling hills and valleys. The sky above Ardenia is ever-changing, painted with hues of crimson, gold, and silver, reflecting the light of countless suns hidden within its depths. The wind howls through the trees, carrying tales of battles long past and dreams yet unfulfilled. In this wild and untamed world, danger lurks at every turn, but so too does opportunity. For those who dare to explore the unknown, there awaits a journey filled with wonder, peril, and perhaps even a glimpse into realms beyond our own.
Surprisedly with this prompt I’ve been able to increase token size to some very large numbers without issue:
$ ./qwen_2.5.py $PROMPT max_token: 5000
In the vast expanse of the Feywild, where magic intertwines with nature, lies a hidden realm known as Whispering Woods. This mystical forest is not just any woodland; it’s a sanctuary of ancient trees, shimmering streams, and ethereal creatures that have been woven into the fabric of the universe itself. The air here is thick with the scent of moss and earth, and the canopy above whispers secrets through the leaves.
As you venture deeper into Whispering Woods, you’ll find yourself surrounded by towering elms that stretch towards the sky like ancient sentinels. These mighty trees are adorned with intricate carvings depicting tales from ages long past, each telling a story of valor and adventure. The ground beneath your feet is soft and moist, covered in wildflowers that bloom in every color imaginable, their petals dancing in the gentle breeze.
The rivers within Whispering Woods are crystal clear and teeming with life. They flow gracefully around the trees, carrying waterfalls that cascade down the slopes, creating cascading pools where fish swim freely among the submerged rocks. At night, these waters glow softly under the stars, mirroring the beauty of the moonlit forest.
In this enchanted landscape, there exists a balance between light and shadow, day and night. The sun rises slowly, painting the sky with hues of gold and crimson, while the shadows dance across the forest floor, casting eerie silhouettes against the luminous branches. During the twilight hours, the woods become alive with the sounds of owls hooting and crickets chirping, adding to the serene yet magical atmosphere.
Whispering Woods is more than just a place; it’s a journey through time and space, a portal to other realms and worlds. It’s a land where the boundaries between reality and imagination blur, making it both a refuge and a gateway to uncharted territories. Whether you’re seeking enlightenment, battling monsters, or simply exploring the unknown, Whispering Woods promises an experience unlike anything else you’ve ever known.
As I mentioned above, these results are really pretty impressive.
However, I wanted to begin exploring the new Qwen3 model trained on 1.7B paramaters, curious to see how it compares in terms of performance, quality, and usability on limited hardware.
One thing I noticed that is different in their example code is max_tokens seems to no longer be needed, and now we are using max_new_tokens:
qwen_3.py
#!/usr/bin/env python
import sys
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "Qwen/Qwen3-1.7B"
# load the tokenizer and the model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype="auto",
device_map="auto"
)
# prepare the model input
if len(sys.argv) == 2:
prompt = sys.argv[1]
else:
prompt = "Give me a short introduction to large language model."
messages = [
{"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
enable_thinking=True # Switches between thinking and non-thinking modes. Default is True.
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
# conduct text completion
generated_ids = model.generate(
**model_inputs,
max_new_tokens=32768
)
output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()
# parsing thinking content
try:
# rindex finding 151668 (</think>)
index = len(output_ids) - output_ids[::-1].index(151668)
except ValueError:
index = 0
thinking_content = tokenizer.decode(output_ids[:index], skip_special_tokens=True).strip("\n")
content = tokenizer.decode(output_ids[index:], skip_special_tokens=True).strip("\n")
print("thinking content:", thinking_content)
print("content:", content)
Using the same prompt as above I’ll ask Qwen3 the same question:
$ ./qwen_3.py $PROMPT
First off I noticed the generation took much longer, and that is because by default Qwen3 performs a <think> step where it reasons before content generation:
Qwen3 models introduce a hybrid approach to problem-solving. They support two modes:
Thinking Mode: In this mode, the model takes time to reason step by step before delivering the final answer. This is ideal for complex problems that require deeper thought. Non-Thinking Mode: Here, the model provides quick, near-instant responses, suitable for simpler questions where speed is more important than depth.
It can provide it’s <think> process verbosely, which honestly is really neat.
thinking content:
Okay, the user wants an engaging introduction for a D&D campaign set in a wilderness area of a high fantasy world. Let me start by visualizing the setting. High fantasy usually has a magical element, so I need to incorporate that. The wilderness should feel vast and mysterious, maybe with ancient ruins or magical creatures. First, the name of the world. Maybe something like “Eldoria” to give it a mystical feel. The wilderness could be called the “Crimson Expanse” to emphasize the dark, magical elements.
I need to describe the environment in detail. The sky might have a different color, like a deep indigo with swirling auroras. The land is covered in forests, maybe with glowing trees. There could be a river that’s magical, like the Veil of Eternity, which has a silver glow.
Inhabitants should be magical beings. Maybe there are ancient spirits or guardians. The creatures could be both dangerous and majestic, like the Luminous Wyrms or the Whispering Wraiths.
The atmosphere should be tense and mysterious. The presence of ancient magic and the unknown dangers. Maybe the wilderness is a place where the old and new worlds meet.
I should include elements that hint at the dangers, like the Veil of Eternity’s dangers, the Whispering Wraiths’ illusions, and the Luminous Wyrms’ presence. Also, the idea of the land being a living entity that watches over the inhabitants.
Need to make sure the introduction is engaging, with vivid descriptions and a sense of adventure. Maybe end with a hook that invites the players into the world. Let me put it all together in a flowing, descriptive way.
Then after the thinking it produces the content:
content: Introducing the Wilds of Eldoria: The Veil of Eternity
In the heart of the High Realm of Eldoria, where the sky is a tapestry of shifting auroras and the earth hums with ancient magic, the Crimson Expanse stretches endlessly—a realm of untamed beauty and peril. Here, the wilds are not merely landscapes but living, breathing entities, shaped by the whispers of forgotten gods and the echoes of a time when the world was young. The air is thick with the scent of pine and ozone, and the ground, beneath your boots, pulses with the faint, rhythmic thrum of magic.
The Crimson Expanse is a mosaic of towering, emerald forests where the trees are not just trees but living sentinels, their roots entwined with veins of silver light that glow faintly in the dark. Above, the sky is a canvas of indigo and violet, streaked with the luminous trails of Aetherwyrms—serpentine dragons of molten gold that soar through the heavens, their scales reflecting the constellations. But the true marvel of this realm lies in the Veil of Eternity, a river of liquid starlight that flows through the heart of the Expanse, its waters shimmering with the remnants of a forgotten age.
Yet, this paradise is not without its shadows. The Whispering Wraiths, spectral beings of wind and shadow, haunt the edges of the forest, their hollow eyes watching from the mist, luring travelers with illusions of forgotten glory. Beyond the trees, the Luminous Wyrms—ancient, colossal dragons that once ruled the land—rest in the craters of the Obsidian Peaks, their presence a reminder of the world’s ancient wars. And in the depths of the Expanse, the Veil of Eternity itself is a double-edged sword: it is both a source of power and a prison, its magic capable of unraveling the fabric of reality if misused.
The inhabitants of this wild realm are as enigmatic as the land itself. The Eldertrees—ancient, sentient beings that guard the forests—communicate through song and visions, their voices a haunting melody that lingers in the wind. The Crimson Keepers, a secretive order of warriors and scholars, dwell in the shadow of the Obsidian Peaks, their knowledge of the Veil’s secrets coveted by both heroes and villains. And in the farthest reaches, the Shattered Veil—a place where time bends, where the past, present, and future collide—beckons those who dare to seek the truth behind the world’s oldest riddles.
As you step into the Wilds of Eldoria, you’ll find yourself at the crossroads of myth and danger. The land is a place of wonder, but it is also a realm of secrets, where every shadow holds a story and every breeze carries a warning. The question remains: will you be the one to uncover the truth, or will the Veil of Eternity claim you as its own?
Welcome to the Wilds of Eldoria. 🌌
This think step can be disabled to speed up generation either in the python or with a /no_think in the prompt, though the response may suffer:
$ ./qwen_3.py "$PROMPT /no_think"
content: In the heart of a vast and untamed wilderness, where the sky is a tapestry of shifting hues and the earth is a living, breathing entity, lies the realm of Eldrithar, a high fantasy world where the boundaries between the natural and the magical blur. Here, ancient forests whisper secrets to the wind, rivers carve paths through stone and time, and the very air hums with the presence of forgotten powers.
Eldrithar is a land of contrasts, where the towering spires of the Crimson Peaks rise like jagged sentinels against the deep blue of the sky, while the Valley of Echoes lies below, a place where time seems to slow, and the echoes of the past linger like a haunting melody. The land is rich with untamed wilderness, where wild creatures roam free, and the inhabitants of Eldrithar are both guardians and wanderers, bound by the ancient will of the world.
In this world, the people are as diverse as the landscapes they inhabit. The Nordic clans of the northern highlands carve their names into the stone, their warriors clad in iron and magic, while the Elven clans of the southern forests weave spells into the very roots of the trees. The Trollkin of the eastern marshes dwell in shadow and stone, their voices a low, rumbling hum that echoes through the depths of the dark glens.
Eldrithar is a world of wonder and peril, where the line between friend and foe is often blurred, and the greatest danger lies not in the monsters that stalk the wild, but in the secrets that lie hidden in the heart of the land itself. It is a realm where heroes are forged in the fires of battle, where magic is both a gift and a curse, and where the fate of the world rests in the hands of those who dare to explore its mysteries.
So, if you are ready to step into the unknown, to face the trials of a world that has waited for you, then Eldrithar awaits—where the wild is wild, the magic is real, and the adventure begins.
And finally lets have a look at those reponse times.
First with 5000 tokens on Qwen2.5:
$ time ./qwen_2.5.py $PROMPT
max_token: 5000
real 0m22.740s
Next with Qwen3 and a think step:
$ time ./qwen_3.py $PROMPT
real 0m56.182s
And finally Qwen3 without a think step:
$ time ./qwen_3.py $POMPT /no_think
real 0m24.417s
Hope you found this as insightful as I have.
Cheers