Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2050 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.8829. Min Act: -0.1700

Data Index: 7267943 (Open Web Text)

Max Activating Token Index: 245

Text #1

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.7083. Min Act: -0.1700

Data Index: 8740090 (Open Web Text)

Max Activating Token Index: 29

Text #2

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.7037. Min Act: -0.1700

Data Index: 5335527 (Open Web Text)

Max Activating Token Index: 385

Text #3

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.6587. Min Act: -0.1700

Data Index: 6135236 (Open Web Text)

Max Activating Token Index: 516

Text #4

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.6381. Min Act: -0.1700

Data Index: 1208629 (Open Web Text)

Max Activating Token Index: 994

Text #5

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.5820. Min Act: -0.1700

Data Index: 8242278 (Open Web Text)

Max Activating Token Index: 749

Text #6

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.4903. Min Act: -0.1700

Data Index: 23934 (Open Web Text)

Max Activating Token Index: 510

Text #7

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.5184. Min Act: -0.1700

Data Index: 1857823 (Open Web Text)

Max Activating Token Index: 293

Text #8

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.4918. Min Act: -0.1700

Data Index: 8714107 (Open Web Text)

Max Activating Token Index: 978

Text #9

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.5614. Min Act: -0.1700

Data Index: 4302925 (Open Web Text)

Max Activating Token Index: 227

Text #10

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.4953. Min Act: -0.1700

Data Index: 357980 (Open Web Text)

Max Activating Token Index: 222

Text #11

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.5019. Min Act: -0.1700

Data Index: 3513169 (Open Web Text)

Max Activating Token Index: 1000

Text #12

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.4561. Min Act: -0.1700

Data Index: 3922669 (Open Web Text)

Max Activating Token Index: 160

Text #13

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.4399. Min Act: -0.1700

Data Index: 4501564 (Open Web Text)

Max Activating Token Index: 1004

Text #14

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.3977. Min Act: -0.1700

Data Index: 3915271 (Open Web Text)

Max Activating Token Index: 534

Text #15

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.3556. Min Act: -0.1700

Data Index: 5873843 (Open Web Text)

Max Activating Token Index: 139

Text #16

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.3848. Min Act: -0.1700

Data Index: 2946818 (Open Web Text)

Max Activating Token Index: 325

Text #17

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.3910. Min Act: -0.1700

Data Index: 6253341 (Open Web Text)

Max Activating Token Index: 464

Text #18

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.3810. Min Act: -0.1700

Data Index: 177515 (Open Web Text)

Max Activating Token Index: 783

Text #19

Max Range: 5.8829. Min Range: -5.8829

Max Act: 5.3704. Min Act: -0.1700

Data Index: 1735795 (Open Web Text)

Max Activating Token Index: 712