Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 54 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.4984. Min Act: -0.1700

Data Index: 260096 (Open Web Text)

Max Activating Token Index: 287

Text #1

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.4225. Min Act: -0.1700

Data Index: 4856536 (Open Web Text)

Max Activating Token Index: 28

Text #2

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.4368. Min Act: -0.1700

Data Index: 5187601 (Open Web Text)

Max Activating Token Index: 98

Text #3

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.4296. Min Act: -0.1700

Data Index: 5048670 (Open Web Text)

Max Activating Token Index: 178

Text #4

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.4388. Min Act: -0.1700

Data Index: 6269940 (Open Web Text)

Max Activating Token Index: 860

Text #5

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.4141. Min Act: -0.1700

Data Index: 5609599 (Open Web Text)

Max Activating Token Index: 540

Text #6

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.4389. Min Act: -0.1700

Data Index: 92623 (Open Web Text)

Max Activating Token Index: 307

Text #7

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.4460. Min Act: -0.1700

Data Index: 2207899 (Open Web Text)

Max Activating Token Index: 120

Text #8

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.3967. Min Act: -0.1700

Data Index: 7523222 (Open Web Text)

Max Activating Token Index: 485

Text #9

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.3913. Min Act: -0.1700

Data Index: 5871299 (Open Web Text)

Max Activating Token Index: 464

Text #10

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.4070. Min Act: -0.1700

Data Index: 801400 (Open Web Text)

Max Activating Token Index: 294

Text #11

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.3897. Min Act: -0.1700

Data Index: 5870373 (Open Web Text)

Max Activating Token Index: 381

Text #12

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.3824. Min Act: -0.1700

Data Index: 2822354 (Open Web Text)

Max Activating Token Index: 149

Text #13

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.4135. Min Act: -0.1700

Data Index: 6281936 (Open Web Text)

Max Activating Token Index: 953

Text #14

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.3899. Min Act: -0.1700

Data Index: 6775738 (Open Web Text)

Max Activating Token Index: 580

Text #15

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.4137. Min Act: -0.1700

Data Index: 5774686 (Open Web Text)

Max Activating Token Index: 214

Text #16

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.3758. Min Act: -0.1700

Data Index: 4236220 (Open Web Text)

Max Activating Token Index: 285

Text #17

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.3801. Min Act: -0.1700

Data Index: 1240008 (Open Web Text)

Max Activating Token Index: 93

Text #18

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.3838. Min Act: -0.1700

Data Index: 1608392 (Open Web Text)

Max Activating Token Index: 929

Text #19

Max Range: 5.4984. Min Range: -5.4984

Max Act: 5.3597. Min Act: -0.1700

Data Index: 6803460 (Open Web Text)

Max Activating Token Index: 382