Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2104 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2122. Min Range: -5.2122

Max Act: 5.2122. Min Act: -0.1700

Data Index: 5788888 (Open Web Text)

Max Activating Token Index: 40

Text #1

Max Range: 5.2122. Min Range: -5.2122

Max Act: 5.1991. Min Act: -0.1700

Data Index: 8354583 (Open Web Text)

Max Activating Token Index: 63

Text #2

Max Range: 5.2122. Min Range: -5.2122

Max Act: 5.1498. Min Act: -0.1700

Data Index: 3318618 (Open Web Text)

Max Activating Token Index: 389

Text #3

Max Range: 5.2122. Min Range: -5.2122

Max Act: 5.0716. Min Act: -0.1700

Data Index: 5089945 (Open Web Text)

Max Activating Token Index: 235

Text #4

Max Range: 5.2122. Min Range: -5.2122

Max Act: 5.0831. Min Act: -0.1700

Data Index: 65199 (Open Web Text)

Max Activating Token Index: 453

Text #5

Max Range: 5.2122. Min Range: -5.2122

Max Act: 5.0961. Min Act: -0.1700

Data Index: 3050325 (Open Web Text)

Max Activating Token Index: 1012

Text #6

Max Range: 5.2122. Min Range: -5.2122

Max Act: 5.0781. Min Act: -0.1700

Data Index: 580874 (Open Web Text)

Max Activating Token Index: 49

Text #7

Max Range: 5.2122. Min Range: -5.2122

Max Act: 5.0615. Min Act: -0.1700

Data Index: 5658219 (Open Web Text)

Max Activating Token Index: 45

Text #8

Max Range: 5.2122. Min Range: -5.2122

Max Act: 5.0765. Min Act: -0.1700

Data Index: 3059888 (Open Web Text)

Max Activating Token Index: 361

Text #9

Max Range: 5.2122. Min Range: -5.2122

Max Act: 5.0541. Min Act: -0.1700

Data Index: 6675531 (Open Web Text)

Max Activating Token Index: 590

Text #10

Max Range: 5.2122. Min Range: -5.2122

Max Act: 5.0565. Min Act: -0.1700

Data Index: 2091342 (Open Web Text)

Max Activating Token Index: 779

Text #11

Max Range: 5.2122. Min Range: -5.2122

Max Act: 5.0135. Min Act: -0.1700

Data Index: 168320 (Open Web Text)

Max Activating Token Index: 204

Text #12

Max Range: 5.2122. Min Range: -5.2122

Max Act: 5.0206. Min Act: -0.1700

Data Index: 8258176 (Open Web Text)

Max Activating Token Index: 111

Text #13

Max Range: 5.2122. Min Range: -5.2122

Max Act: 5.0079. Min Act: -0.1700

Data Index: 6731755 (Open Web Text)

Max Activating Token Index: 307

Text #14

Max Range: 5.2122. Min Range: -5.2122

Max Act: 5.0338. Min Act: -0.1700

Data Index: 4469766 (Open Web Text)

Max Activating Token Index: 760

Text #15

Max Range: 5.2122. Min Range: -5.2122

Max Act: 5.0267. Min Act: -0.1700

Data Index: 3780433 (Open Web Text)

Max Activating Token Index: 819

Text #16

Max Range: 5.2122. Min Range: -5.2122

Max Act: 5.0117. Min Act: -0.1700

Data Index: 408477 (Open Web Text)

Max Activating Token Index: 453

Text #17

Max Range: 5.2122. Min Range: -5.2122

Max Act: 4.9982. Min Act: -0.1700

Data Index: 7353572 (Open Web Text)

Max Activating Token Index: 269

Text #18

Max Range: 5.2122. Min Range: -5.2122

Max Act: 4.9820. Min Act: -0.1700

Data Index: 7196872 (Open Web Text)

Max Activating Token Index: 675

Text #19

Max Range: 5.2122. Min Range: -5.2122

Max Act: 5.0077. Min Act: -0.1700

Data Index: 2784832 (Open Web Text)

Max Activating Token Index: 561