Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 832 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.2717. Min Act: -0.1700

Data Index: 8395605 (Open Web Text)

Max Activating Token Index: 741

Text #1

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.2441. Min Act: -0.1700

Data Index: 6465352 (Open Web Text)

Max Activating Token Index: 632

Text #2

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.2059. Min Act: -0.1700

Data Index: 8129859 (Open Web Text)

Max Activating Token Index: 222

Text #3

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.1197. Min Act: -0.1700

Data Index: 6553658 (Open Web Text)

Max Activating Token Index: 659

Text #4

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.1222. Min Act: -0.1700

Data Index: 2613285 (Open Web Text)

Max Activating Token Index: 189

Text #5

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.1020. Min Act: -0.1700

Data Index: 8734016 (Open Web Text)

Max Activating Token Index: 173

Text #6

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.0480. Min Act: -0.1700

Data Index: 5259558 (Open Web Text)

Max Activating Token Index: 320

Text #7

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.0847. Min Act: -0.1700

Data Index: 1898377 (Open Web Text)

Max Activating Token Index: 776

Text #8

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.0228. Min Act: -0.1700

Data Index: 3741225 (Open Web Text)

Max Activating Token Index: 392

Text #9

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.0508. Min Act: -0.1700

Data Index: 8531942 (Open Web Text)

Max Activating Token Index: 548

Text #10

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.0191. Min Act: -0.1700

Data Index: 5327472 (Open Web Text)

Max Activating Token Index: 313

Text #11

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.0258. Min Act: -0.1700

Data Index: 4905593 (Open Web Text)

Max Activating Token Index: 224

Text #12

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.0286. Min Act: -0.1700

Data Index: 5941922 (Open Web Text)

Max Activating Token Index: 13

Text #13

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.0273. Min Act: -0.1700

Data Index: 2195316 (Open Web Text)

Max Activating Token Index: 22

Text #14

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.0416. Min Act: -0.1700

Data Index: 4743250 (Open Web Text)

Max Activating Token Index: 946

Text #15

Max Range: 4.2717. Min Range: -4.2717

Max Act: 3.9916. Min Act: -0.1700

Data Index: 3847 (Open Web Text)

Max Activating Token Index: 43

Text #16

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.0226. Min Act: -0.1700

Data Index: 6740000 (Open Web Text)

Max Activating Token Index: 252

Text #17

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.0070. Min Act: -0.1700

Data Index: 6677135 (Open Web Text)

Max Activating Token Index: 889

Text #18

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.0041. Min Act: -0.1700

Data Index: 4669046 (Open Web Text)

Max Activating Token Index: 146

Text #19

Max Range: 4.2717. Min Range: -4.2717

Max Act: 4.0073. Min Act: -0.1700

Data Index: 8611070 (Open Web Text)

Max Activating Token Index: 485