Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1230 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.9968. Min Act: -0.1700

Data Index: 8106777 (Open Web Text)

Max Activating Token Index: 13

Text #1

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.9085. Min Act: -0.1700

Data Index: 1463405 (Open Web Text)

Max Activating Token Index: 154

Text #2

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.8932. Min Act: -0.1700

Data Index: 4979645 (Open Web Text)

Max Activating Token Index: 573

Text #3

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.8948. Min Act: -0.1700

Data Index: 5060051 (Open Web Text)

Max Activating Token Index: 46

Text #4

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.9092. Min Act: -0.1700

Data Index: 2431148 (Open Web Text)

Max Activating Token Index: 6

Text #5

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.8839. Min Act: -0.1700

Data Index: 1570727 (Open Web Text)

Max Activating Token Index: 360

Text #6

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.8861. Min Act: -0.1700

Data Index: 4291415 (Open Web Text)

Max Activating Token Index: 412

Text #7

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.8859. Min Act: -0.1700

Data Index: 8177416 (Open Web Text)

Max Activating Token Index: 138

Text #8

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.8543. Min Act: -0.1700

Data Index: 5138134 (Open Web Text)

Max Activating Token Index: 895

Text #9

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.8587. Min Act: -0.1700

Data Index: 2024308 (Open Web Text)

Max Activating Token Index: 860

Text #10

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.8422. Min Act: -0.1700

Data Index: 822190 (Open Web Text)

Max Activating Token Index: 21

Text #11

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.8473. Min Act: -0.1700

Data Index: 5560226 (Open Web Text)

Max Activating Token Index: 51

Text #12

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.8546. Min Act: -0.1700

Data Index: 3977017 (Open Web Text)

Max Activating Token Index: 370

Text #13

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.8523. Min Act: -0.1700

Data Index: 8184587 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.8561. Min Act: -0.1700

Data Index: 6967345 (Open Web Text)

Max Activating Token Index: 364

Text #15

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.8535. Min Act: -0.1700

Data Index: 1837224 (Open Web Text)

Max Activating Token Index: 269

Text #16

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.8484. Min Act: -0.1700

Data Index: 6205863 (Open Web Text)

Max Activating Token Index: 920

Text #17

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.8375. Min Act: -0.1700

Data Index: 890950 (Open Web Text)

Max Activating Token Index: 807

Text #18

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.8369. Min Act: -0.1700

Data Index: 1371035 (Open Web Text)

Max Activating Token Index: 6

Text #19

Max Range: 2.9968. Min Range: -2.9968

Max Act: 2.8374. Min Act: -0.1700

Data Index: 5547329 (Open Web Text)

Max Activating Token Index: 13