Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2201 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.6109. Min Act: -0.1700

Data Index: 8026077 (Open Web Text)

Max Activating Token Index: 998

Text #1

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.4756. Min Act: -0.1700

Data Index: 1790983 (Open Web Text)

Max Activating Token Index: 315

Text #2

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.4692. Min Act: -0.1700

Data Index: 5135716 (Open Web Text)

Max Activating Token Index: 38

Text #3

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.4678. Min Act: -0.1700

Data Index: 3616675 (Open Web Text)

Max Activating Token Index: 57

Text #4

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.4706. Min Act: -0.1700

Data Index: 420990 (Open Web Text)

Max Activating Token Index: 473

Text #5

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.4522. Min Act: -0.1700

Data Index: 8309552 (Open Web Text)

Max Activating Token Index: 570

Text #6

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.4322. Min Act: -0.1700

Data Index: 6878688 (Open Web Text)

Max Activating Token Index: 64

Text #7

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.4078. Min Act: -0.1700

Data Index: 5135717 (Open Web Text)

Max Activating Token Index: 105

Text #8

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.3944. Min Act: -0.1700

Data Index: 659147 (Open Web Text)

Max Activating Token Index: 704

Text #9

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.4165. Min Act: -0.1700

Data Index: 5012251 (Open Web Text)

Max Activating Token Index: 400

Text #10

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.4015. Min Act: -0.1700

Data Index: 1321072 (Open Web Text)

Max Activating Token Index: 778

Text #11

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.4342. Min Act: -0.1700

Data Index: 3038409 (Open Web Text)

Max Activating Token Index: 801

Text #12

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.3631. Min Act: -0.1700

Data Index: 3124847 (Open Web Text)

Max Activating Token Index: 354

Text #13

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.4245. Min Act: -0.1700

Data Index: 5682485 (Open Web Text)

Max Activating Token Index: 405

Text #14

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.3977. Min Act: -0.1700

Data Index: 5852319 (Open Web Text)

Max Activating Token Index: 757

Text #15

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.4043. Min Act: -0.1700

Data Index: 6238975 (Open Web Text)

Max Activating Token Index: 172

Text #16

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.3759. Min Act: -0.1700

Data Index: 4973470 (Open Web Text)

Max Activating Token Index: 546

Text #17

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.3592. Min Act: -0.1700

Data Index: 5530229 (Open Web Text)

Max Activating Token Index: 15

Text #18

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.3627. Min Act: -0.1700

Data Index: 1672665 (Open Web Text)

Max Activating Token Index: 11

Text #19

Max Range: 4.6109. Min Range: -4.6109

Max Act: 4.3637. Min Act: -0.1700

Data Index: 5592407 (Open Web Text)

Max Activating Token Index: 698