Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3063 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.6791. Min Act: -0.1700

Data Index: 6507725 (Open Web Text)

Max Activating Token Index: 302

Text #1

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.6047. Min Act: -0.1700

Data Index: 1693078 (Open Web Text)

Max Activating Token Index: 755

Text #2

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.5833. Min Act: -0.1700

Data Index: 8601257 (Open Web Text)

Max Activating Token Index: 889

Text #3

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.5307. Min Act: -0.1700

Data Index: 3203077 (Open Web Text)

Max Activating Token Index: 385

Text #4

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.5398. Min Act: -0.1700

Data Index: 3585476 (Open Web Text)

Max Activating Token Index: 275

Text #5

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.5430. Min Act: -0.1700

Data Index: 444894 (Open Web Text)

Max Activating Token Index: 442

Text #6

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.4975. Min Act: -0.1700

Data Index: 1503640 (Open Web Text)

Max Activating Token Index: 266

Text #7

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.4914. Min Act: -0.1700

Data Index: 6940626 (Open Web Text)

Max Activating Token Index: 643

Text #8

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.4785. Min Act: -0.1700

Data Index: 4266957 (Open Web Text)

Max Activating Token Index: 443

Text #9

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.4752. Min Act: -0.1700

Data Index: 6040677 (Open Web Text)

Max Activating Token Index: 385

Text #10

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.4420. Min Act: -0.1700

Data Index: 6894277 (Open Web Text)

Max Activating Token Index: 317

Text #11

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.4374. Min Act: -0.1700

Data Index: 7152983 (Open Web Text)

Max Activating Token Index: 600

Text #12

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.4392. Min Act: -0.1700

Data Index: 3537191 (Open Web Text)

Max Activating Token Index: 877

Text #13

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.4197. Min Act: -0.1700

Data Index: 3856550 (Open Web Text)

Max Activating Token Index: 676

Text #14

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.4234. Min Act: -0.1700

Data Index: 4459128 (Open Web Text)

Max Activating Token Index: 240

Text #15

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.4243. Min Act: -0.1700

Data Index: 5545450 (Open Web Text)

Max Activating Token Index: 301

Text #16

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.4161. Min Act: -0.1700

Data Index: 7915433 (Open Web Text)

Max Activating Token Index: 529

Text #17

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.4241. Min Act: -0.1700

Data Index: 77879 (Open Web Text)

Max Activating Token Index: 97

Text #18

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.3993. Min Act: -0.1700

Data Index: 6533493 (Open Web Text)

Max Activating Token Index: 494

Text #19

Max Range: 3.6791. Min Range: -3.6791

Max Act: 3.4054. Min Act: -0.1700

Data Index: 1932174 (Open Web Text)

Max Activating Token Index: 634