Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 568 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.6128. Min Act: -0.1700

Data Index: 6874126 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.5610. Min Act: -0.1700

Data Index: 4247921 (Open Web Text)

Max Activating Token Index: 7

Text #2

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.5378. Min Act: -0.1700

Data Index: 125066 (Open Web Text)

Max Activating Token Index: 3

Text #3

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.5395. Min Act: -0.1700

Data Index: 7662960 (Open Web Text)

Max Activating Token Index: 2

Text #4

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.5391. Min Act: -0.1700

Data Index: 3790635 (Open Web Text)

Max Activating Token Index: 6

Text #5

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.5167. Min Act: -0.1700

Data Index: 3906117 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.5240. Min Act: -0.1700

Data Index: 689636 (Open Web Text)

Max Activating Token Index: 5

Text #7

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.5046. Min Act: -0.1700

Data Index: 3069484 (Open Web Text)

Max Activating Token Index: 6

Text #8

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.5180. Min Act: -0.1700

Data Index: 1782192 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.4966. Min Act: -0.1700

Data Index: 4996287 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.5213. Min Act: -0.1700

Data Index: 7947397 (Open Web Text)

Max Activating Token Index: 6

Text #11

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.5197. Min Act: -0.1700

Data Index: 7731652 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.5196. Min Act: -0.1700

Data Index: 6945270 (Open Web Text)

Max Activating Token Index: 4

Text #13

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.5108. Min Act: -0.1700

Data Index: 5766869 (Open Web Text)

Max Activating Token Index: 4

Text #14

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.4805. Min Act: -0.1700

Data Index: 3438218 (Open Web Text)

Max Activating Token Index: 4

Text #15

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.4810. Min Act: -0.1700

Data Index: 1759912 (Open Web Text)

Max Activating Token Index: 6

Text #16

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.4960. Min Act: -0.1700

Data Index: 4105268 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.4919. Min Act: -0.1700

Data Index: 712922 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.4960. Min Act: -0.1700

Data Index: 1802164 (Open Web Text)

Max Activating Token Index: 3

Text #19

Max Range: 3.6128. Min Range: -3.6128

Max Act: 3.4632. Min Act: -0.1700

Data Index: 4318256 (Open Web Text)

Max Activating Token Index: 5