Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2223 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3191. Min Range: -3.3191

Max Act: 3.3191. Min Act: -0.1700

Data Index: 2325484 (Open Web Text)

Max Activating Token Index: 428

Text #1

Max Range: 3.3191. Min Range: -3.3191

Max Act: 3.2498. Min Act: -0.1700

Data Index: 2325483 (Open Web Text)

Max Activating Token Index: 947

Text #2

Max Range: 3.3191. Min Range: -3.3191

Max Act: 2.9593. Min Act: -0.1700

Data Index: 3994147 (Open Web Text)

Max Activating Token Index: 876

Text #3

Max Range: 3.3191. Min Range: -3.3191

Max Act: 2.9673. Min Act: -0.1700

Data Index: 4848782 (Open Web Text)

Max Activating Token Index: 621

Text #4

Max Range: 3.3191. Min Range: -3.3191

Max Act: 2.9235. Min Act: -0.1700

Data Index: 8233144 (Open Web Text)

Max Activating Token Index: 486

Text #5

Max Range: 3.3191. Min Range: -3.3191

Max Act: 2.9338. Min Act: -0.1698

Data Index: 8233143 (Open Web Text)

Max Activating Token Index: 793

Text #6

Max Range: 3.3191. Min Range: -3.3191

Max Act: 2.9318. Min Act: -0.1700

Data Index: 1901435 (Open Web Text)

Max Activating Token Index: 760

Text #7

Max Range: 3.3191. Min Range: -3.3191

Max Act: 2.9017. Min Act: -0.1700

Data Index: 3430610 (Open Web Text)

Max Activating Token Index: 953

Text #8

Max Range: 3.3191. Min Range: -3.3191

Max Act: 2.9009. Min Act: -0.1696

Data Index: 8233142 (Open Web Text)

Max Activating Token Index: 540

Text #9

Max Range: 3.3191. Min Range: -3.3191

Max Act: 2.9083. Min Act: -0.1700

Data Index: 2711830 (Open Web Text)

Max Activating Token Index: 160

Text #10

Max Range: 3.3191. Min Range: -3.3191

Max Act: 2.8846. Min Act: -0.1694

Data Index: 8233140 (Open Web Text)

Max Activating Token Index: 458

Text #11

Max Range: 3.3191. Min Range: -3.3191

Max Act: 2.8938. Min Act: -0.1700

Data Index: 6345854 (Open Web Text)

Max Activating Token Index: 742

Text #12

Max Range: 3.3191. Min Range: -3.3191

Max Act: 2.8899. Min Act: -0.1700

Data Index: 7194018 (Open Web Text)

Max Activating Token Index: 738

Text #13

Max Range: 3.3191. Min Range: -3.3191

Max Act: 2.8703. Min Act: -0.1700

Data Index: 5628522 (Open Web Text)

Max Activating Token Index: 57

Text #14

Max Range: 3.3191. Min Range: -3.3191

Max Act: 2.8559. Min Act: -0.1700

Data Index: 3979263 (Open Web Text)

Max Activating Token Index: 671

Text #15

Max Range: 3.3191. Min Range: -3.3191

Max Act: 2.8395. Min Act: -0.1700

Data Index: 1159210 (Open Web Text)

Max Activating Token Index: 518

Text #16

Max Range: 3.3191. Min Range: -3.3191

Max Act: 2.8470. Min Act: -0.1700

Data Index: 4285588 (Open Web Text)

Max Activating Token Index: 605

Text #17

Max Range: 3.3191. Min Range: -3.3191

Max Act: 2.8482. Min Act: -0.1700

Data Index: 981096 (Open Web Text)

Max Activating Token Index: 386

Text #18

Max Range: 3.3191. Min Range: -3.3191

Max Act: 2.8192. Min Act: -0.1700

Data Index: 6169589 (Open Web Text)

Max Activating Token Index: 532

Text #19

Max Range: 3.3191. Min Range: -3.3191

Max Act: 2.8620. Min Act: -0.1700

Data Index: 3928878 (Open Web Text)

Max Activating Token Index: 983