Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 485 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.5410. Min Act: -0.1700

Data Index: 315329 (Open Web Text)

Max Activating Token Index: 950

Text #1

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.5239. Min Act: -0.1700

Data Index: 5765043 (Open Web Text)

Max Activating Token Index: 528

Text #2

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.5035. Min Act: -0.1700

Data Index: 3677965 (Open Web Text)

Max Activating Token Index: 926

Text #3

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.4627. Min Act: -0.1700

Data Index: 3440954 (Open Web Text)

Max Activating Token Index: 965

Text #4

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.4648. Min Act: -0.1700

Data Index: 7797229 (Open Web Text)

Max Activating Token Index: 763

Text #5

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.4764. Min Act: -0.1700

Data Index: 7519064 (Open Web Text)

Max Activating Token Index: 600

Text #6

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.4803. Min Act: -0.1700

Data Index: 7066442 (Open Web Text)

Max Activating Token Index: 864

Text #7

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.4509. Min Act: -0.1700

Data Index: 5765044 (Open Web Text)

Max Activating Token Index: 87

Text #8

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.4556. Min Act: -0.1700

Data Index: 1737934 (Open Web Text)

Max Activating Token Index: 461

Text #9

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.4655. Min Act: -0.1700

Data Index: 4437624 (Open Web Text)

Max Activating Token Index: 752

Text #10

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.4446. Min Act: -0.1700

Data Index: 3672826 (Open Web Text)

Max Activating Token Index: 660

Text #11

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.4404. Min Act: -0.1700

Data Index: 1869324 (Open Web Text)

Max Activating Token Index: 126

Text #12

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.3989. Min Act: -0.1700

Data Index: 8610340 (Open Web Text)

Max Activating Token Index: 594

Text #13

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.4275. Min Act: -0.1700

Data Index: 2971584 (Open Web Text)

Max Activating Token Index: 791

Text #14

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.3933. Min Act: -0.1700

Data Index: 1448837 (Open Web Text)

Max Activating Token Index: 537

Text #15

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.4025. Min Act: -0.1700

Data Index: 3713172 (Open Web Text)

Max Activating Token Index: 1022

Text #16

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.3973. Min Act: -0.1700

Data Index: 574 (Open Web Text)

Max Activating Token Index: 207

Text #17

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.3746. Min Act: -0.1700

Data Index: 8247612 (Open Web Text)

Max Activating Token Index: 451

Text #18

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.3531. Min Act: -0.1700

Data Index: 1157099 (Open Web Text)

Max Activating Token Index: 700

Text #19

Max Range: 4.5410. Min Range: -4.5410

Max Act: 4.3532. Min Act: -0.1700

Data Index: 626099 (Open Web Text)

Max Activating Token Index: 463